I'll chuck my hat in as a moderately quick rider...
Best way to approach being passed is to keep riding but just move off the racing line slightly if possible, the amount of people that (generously) throw themselves into the undergrowth or slam on the anchors when you call to pass is amazing! As a passing rider, I normally have momentum and have allowed for the fact you are moving, when you suddenly stop, I have to make a quicker decision how to deal with it, also as the passer I have to benefit of being able to see what's going on, the rider in front doesn't have eyes in the back of their head so can't see what's going on as easily.
I generally call from a fair way back to warn and then call again to let you know what side when I'm a bit closer, sometimes it may be a while before I want to pass anyway, we get knackered too!! Most of the time there's no need to do anything as I'd generally make it my issue to get past without affecting your race. Most real quick riders will be the same, we've overtaken a fair few riders in our time so get ok at doing it, as most people say, it's the mid packers that will present the trouble if there is any.
Don't forget this doesn't apply to people in your own race, you only need to give track position to other faster categories so that you don't affect a different race, guy's you're racing in the same cat are behind you for a reason!
Don't use headphones, they're banned. You won't need 'em anyway, you're gonna be racing, not riding. You'll need to use every sense you've got available. Have a bit of banter with your fellow racers, the amount of miserable feckers I see is amazing, you're doing this out of choice!