It all depends on how warm you get, and how cold it is, and the weight of fleece you’re wearing, how tired you tired you are and how much you’ve eaten and when can affect how warm you feel too.
That said, a thinner weight fleece can be just the thing for me when it’s cold enough for there to be snow and I’m not riding ‘full burn’, but more pottering along at the kind of pace where I could just about chat, and might have a Howies Waffle mid layer and a merino base layer underneath, with a wind/water proof on top. Through unzipping the zips when too hot, the lot can stay on and be all zipped up if it gets colder or I stop, so it can save fiddling with my bag, it has to be towards the ‘colder end of cold’ though.
A thicker weight gillet or body warmer can be just the thing for adding some core warmth without getting too hot, I bought a Regatta one for five pounds in CCC in Sheffield before it became Go Outdoors, and it’s one of my better impulse buys.
It’s probably a good idea to take a back up layer of warmth if it’s snow-cold.
Charity shops can be a cost effective way of buying fleeces to experiment with.