I ALWAYS thank the polite ones.
This, is not only a great habit as it is the decent thing to do, but also the most perfectly targeted weapon against the diehard anti mtb rambler types.
I’m always incredibly nice, polite, slightly chatty and generally lovely when I sense a grumbly red socker in a group. And they HATE it. They want a mtber who complies to the stereotype of rude and dangerous, so they can bitch on about them for the rest of the walk, tell people in the pub afterwards about how he was damn nearly killed etc.
But if you’re really nice, give way, or give them a polite warning, give a comment about the weather, bob off the track to pass them etc, you deny them that and contradict their opinion.
And to anyone not of that mindset, it just seems like your being nice. No collateral damage unlike getting into an arguement.
Watch their faces, you can see it work, its wonderful :D