For what purpose?
Common wording is ‘to pass and repass’. In the absence of limiting wording – like, as you say, solely for the purposes of maintenance – then it is usually construed as being for any reasonable purpose.
Language is also often interpreted in the context of the time it was written – so, for example, reference to ‘handbarrows’ was used to limit vehicular access along a track behind some houses, but access for ‘coaches and carts’ was deemed to allow vehicular usage as those were the common methods of transportation at the time the right was granted.
If the OP had decided to conduct coach tours through the gardens to use his facilities in return for payment (for example) then that might constitute an unreasonable exercise of the right (so-called ‘excessive user’).
That said, even if the wording was limited to ‘solely for use to put the bins out’, it wouldn’t allow the neighbour to restrict access.