Whilst I do understand that there are many factors to your OP. I do have to say that considering, and reluctantly or not, they have been a bit flexible with your hours,which is a significant benefit that for many is simply not possible. As your Manager / Boss the claiming the 2 hours thing would have pissed me off too.
EDIT: Being flexible has to work BOTH ways.