Hi I broke my ankle in two places a few years ago, they removed my plaster after four weeks to allow me to go on a pre booked holiday, while away I did as much swimming as possible and it really helped, after 7-8 weeks I tried riding with flats four a few weeks then went back to spds and found it ok, if your still getting pain I’d give it a few more weeks to be on the safe side and see how it goes.