Forget the obsession with down and go synthetic if its for post riding wearing. Chances are you will be wet and sweaty, mud and water will get on you somehow putting kit or bikes in the car, and its probably going to be raining.
In this situation ive been using a synthetic filled jacket for a few years, its been superb. Get it covered in mud, rained on, even put it on over sweaty base layers when you are freezing and faffing before getting time to get changed. The synthetic means it works better when wet and I dont need to worry about it at all.
The one ive got is a proto version of the Alpkit 0Hiro (pronounced zero-hero) belay jacket, really really good jacket. Ive also got a big down Alpkit jacket, a PHD down jacket and a Howies down jacket. None of them offer what the 0Hiro does, its my go-to jacket.