I tend to ride in non-cycling specific shorts (sometimes with lycra underneath) or stretch jeans, with t-shirt (and hoodie, and often other thin jumpers to give me extra layers). I buy jeans specifically for cycling.
I tend to wear a BMX-style helmet for XC, though sometimes I put on a ‘normal’ roadie / MTB helmet).
I work on the basis that –
(1) Cycling specific clothing is not necessary
(2) Cycling specific clothing is ugly
(3) I will be a dirty, sweaty mess at the end of the ride no matter what I’m wearing. Unless I am going really really slow in which case why not wear ‘normal’ clothes?
(4) I am not racing so the ‘fastest’ clothing is not a consideration
FWIW I take the same view on clothing whether on a BMX, a road bike or an old retro steel lo pro time trial bike. Only difference is MTB SPDs and 661 trainer-like SPD shoes on road bikes and BMX trainers and flats on non-road bikes! One day I might go back to SPDs on MTBs, but I can’t see it.
To me cycling is ‘pissing about on a bike for a laugh’ and you really don’t need lycra for that.