A few of us were musing over the difference between the really fast riders at Bonty 24 (& the rest of us!) & wondered about the size of peoples lungs.
It's not quite that, but you're not far off. Google "VO2max", and see the effect of having a massively efficient "engine" is on performance. Couple this with the ability to turn out high power (which is essentially a function of VO2, as I understand), and you're bound for glory.
The key factor in whether someone is destined for the top or the bottom is dictated by natural physiology. Sure, you can train that, but having the basis inately in you is what's needed.