I am still not convinced on the roadie fitness thing…
2 hours constant XC will use far more energy than 2 hours constant road riding. The road bike covers greater distances with less resistance!
As for
If we use Strava as a guide, i am one of the fastest mtbers in my area, but average as a roadie.
simply because there are excellent riders in your area who don’t ride MTB.
I use the road bike to enhance my endurance fitness (what there is of it) as I know I will sustain a steadier HR for longer.
I am not knocking Roadies or their fitness, I think it is a different kind of fitness, and even Boardman commentating at the olympics waxed lyrical about how fit the XC racers were as they rode with consistently higher heartrates.
As for the OP wear what you like, ride what you like, joining a club will get you fitter quicker and most clubs encourage beginners no matter who they are!