What's more important is your body type on the inside which is related to what you look like to an extent. For mtbing sustainable power to weight ratio is what it's about since you spend most of your effort redlining it up climbs. It's true most super fast XC riders seem to be small skinny guys (low power but lower weight) although there are some bigger quicker guys. The only big men you seem to see on telly cycling are track cyclists.
What Handsomedog says is right about muscle fibres, but it's worth noting that a 'short' mtb race is far longer than many so-called endurance events in other sports. People with fast twitch tend to be the type that have some meat on their bones, but sometimes can also be prone to putting on fat as well as muscle. For example the big fat bloke on a Patriot who burned me off up some climbs in an XC race years ago when I only weighed about 12 stone.
It's not cut and dried in MTBing like it is in say distance running tho, partly due to the mix of courses. In a flatter course like the one near Ringwood (Crow Hill is it?) I have a ball and do fairly well since I can put loads of power down on flat sections, and I have the upper body muscle to hammer technically. I know skinny buggers who hate Thetford cos it's unrelenting – I love it cos there's so little climbing! That's not to say I don't enjoy climbing as a natural sprinter/meso/endomprh but I just don't do it as quickly as skinnier folk.