Hi,
I broke my wrist (schaphoid) about two years ago now. I broke it falling off my bike on ice during a solo night ride – bit silly I know.
I broke it on the Monday night and had it in a cast and was riding (small) dirt jumps on the Saturday of the same week. I had to file the cast down a bit to be able to grab the bars but it worked out ok. Went street riding on the sunday (which hurt a bit) but pretty much rode normal trails ok with the cast on for a few weeks.
It seemed to heal pretty quickly and now it is ‘almost’ as good as my un-broken wrist. I can still feel it a bit when carrying heavy shopping bags in one hand but its not a real issue.
I don’t think I caused any more harm by riding too soon as most people seem to say any broken bone never feels as good as before.