I’m a soft-handed keyboard-botherer who likes a bit of controlled suffering, but tends to back off rather than going all out every ride. Personally I don’t really think there is a correlation, you’ve only got to look at how many posties or soldiers also love a bit of cycling.
A lot of the most prolific DH and BMX trail builders also do physical day jobs, even though you might think that getting busy with a spade is a busman’s holiday for them. We had one guy come to a dig day who is a landscape gardener and a sponsored masters DH racer, and I’ve never seen anyone work as hard.
Maybe it’s that the ones who are fittest due to not sitting down for eight hours a day tend to make riding look easy and bang on about it less? 😉