I don't ever really seem to have this problem - must just be something you lot are doing!
I guess you must just be riding where there are no pedestrians. Over the years I've been riding the local Sustrans network, which is pretty much since I bough my first mountain bike, in 1988, I've had every experience given above.
I've even been told off for using a bell because it's impolite!
I've been told off for going too fast when I had to make a sudden grab of the brakes when an off-the-lead dog suddenly wandered in front, as I was about to pass the owner; I'd actually rung my bell, and been forced to practically track-stand behind him as he dawdled along, oblivious to my presence three feet behind him!
Then there's the group of four walking abreast in front, who turned and stared in a bovine fashion, after I was forced to call, very loudly, 'excuse me!', when they ignored my rather loud bell I'd been ringing from thirty or forty yards behind them. 'Oh, sorry, didn't hear you. You should use a bell' AAAARRRRRRGGGGGHHHHHH!
As a volunteer Sustrans Ranger, I have to be careful how I react, but the temptation, oh, the temptation...