I purchased a Ragley Big Al 2 (2021) in size large, turned out to be slightly too small after a few rides. Instead of selling the now used bike I purchased the same frame in XL and swapped all the original parts over. Now the XL head tube (120mm) is 10mm higher than the large (110mm). I removed the headset cups and pressed them into the new frame and swapped the headset bearings and spacers over etc. All seems to fit correctly and well but I can't figure if I'm now imagining some micro play in the headset. I can't see any issues with what I have done, the fork & headset should be the same for both frame sizes, right ?
http://ragleybikes.com/products/big-al-2-0-complete-bike-2021/
....
Fork steerer will likely be shorter on the large, so you'll have fewer spacers to play with. My guess is top cap fouling the steerer tube resulting in a micro amount of play. Stick another spacer below the top cap and see if that solves it.