My Olympus caps are prone to coming off if I don’t make sure they’re fully clipped into the last groove of the filter thread. If I don’t quite find the last one it falls off quite easily.
Have you considered screw lens caps? Assuming your lens has a filter thread. My 25mm pancake hs a lovely super slim machined aluminium number which can’t really fall off. It’s not that cumbersome to use either, not as much as I thought it would be.
My wife has managed to lose her lens cap in the spare room. When I find it I’m going to drill a hole in it and fit a bit of string.