I’m in the camp of “every gram counts” as much as I can afford.
I don’t think a 100g heavier saddle makes any difference to my riding or enjoyment at all. I do believe that all of those extra grams adds up to a big dfference as a complete bike though.
I weighed two bikes last week and found one to be 4.5kgs heavier than the other. That’s a big and noticeable difference. The frames were the same weight on their own and set-up for XC riding…
It’s always a compromise though. I got a steel middle chainring as it lasts longer, and I’ll never remove my bar ends because I like the comfort. By the same token I save weight by having lighter tyres and tubes that aren’t filled with slime.
Given limited money though I’d spend the most money on trying to save weight on the frame, then wheels and tyres, and then forks.