I have ridden flat pedals on all of my bikes since I started riding. I used some clip in pedals on my road bike for the first time earlier, felt absolutely bizarre.

Had to give up towards the end and pootle home because it felt like my right knee was going to burst.

Is it just a case of riding them for a while and my muscles adjusting?

Any tips for setting them up?

They take time to get used to from a technique point of view but they shouldnt hurt.

I found my right foot wouldnt clip in as easy as the the left and my right knee hurt a bit. I just needed to push the cleet over a bit and its been great since.

I have wonkey legs 🙂

I know some people never get on with them though

No your knees should not hurt you need to play around with the cleat position until you find the natural position your foot sits at .The float that the cleat has should allow the sideways movement you need so your knees are not knackered by being forced into an unnatural position

Instantly, I found no issues (Time attak). Maybe, with being clipped in, a slight concern on technical rough stuff, to start off with.

I used SPD's for the first time 6-7 years ago. Went back to flats after a few months, but now I'm using SPD's again! I can't see myself ever going back to flats.
Start off with as much float as possible and over time, tighten them bit by bit until you feel comfortable.
Also, the position of the cleats on your shoes could need adjusting.
I moved my saddle forwards slightly and this took the slight pain away from my knees that I was having when using SPD's.
It does take a while to get used to them, but it's worth persevering IMO.

3 rides, 3 tumbles per ride, none after.

You may have to alter saddle height as well as clippy shoes/pedal combo can be quite tall

I fitted SPDs to my road bike, with a view to fitting them to the MTB later on.

Absolutely fine on the roadie, I've done two rides on the MTB and I'm sacking it off.

The problem for me is the total lack of confidence engendered by the SPDs on even remotely technical terrain. Whereas before I'd happily barrel through on flats, I slow down, mince, and ultimately unclip and stop before I actually attack the trail feature.

Not for me.

May work for you though.

A few ride. There should be no pain.

I had a similar problem when changing cleats on my road bike a while back; if you're using road style chunky plastic cleats check they have float – the ones that gave me grief were black look cleats (no float) red ones have plenty and pain went away.