I had this issue a few years ago.
Spoke to my insurance company who said I would be fine to drive IF I got a letter from my Doctor saying I could drive. Spoke to the Doctor who said he wouldn't write a letter and it was up to the insurance company. So I ended up stuck between the two and ended up not driving.
Also broke my wrist about 15 years ago (it seems to be a recurring theme in my life) and was a witness to an accident where the police were called. I was asked by the Police if I was driving, which I said I was and they weren't fussed about the cast on my arm.
So I don't think the Police care, unless it causes an accident, but if you have a problem I bet your insurance company will use it as an excuse not to pay out.