Speaking as a cycle mechanic, the worst job is working on a bike that is so badly engineered that all the skills in the world won’t get it to a satisfactory condition. As mentioned above then – working on bike shaped objects.
As an after thought, working on bikes that’ll never be used, or bikes owned by a-holes is pretty demoralizing also. Conversely working on bike belonging to customers who buy me doughnuts etc, is one of lifes pleasures.
Personally and maybe irrationally – I hate bolting the back wheels into singlespeeds/fixies/bmx’s and the axle shifts in the dropouts as you torque the bolts. That rips my knitting. All bikes should have tugs, or what I call Ritchey style dropouts and eccentric BB’s. It’s the law round here (in my microcosm).