Don't worry about it – you'll only increase your HR!
Whenever testing my (resting) HR I was suprised that I was always pretty high, around 70 and even 80, even though I was pretty dang fit (training every day). Wife took it when I was asleep and it was waaaaay lower 40 or something – soon as I woke up it shot up (hmmm prehaps it has something to do with her)!
You may just be a 'high beater' (so to speak). I use HRM to train, but not bothered about trying to lower it, just see how it changes.
Wore it in an event once – that was waaay high just from the adrenaline of doing the event I imagine as we were pootling along.