Possibly two reasons.
If you drop weight for a peak race this may not be sustainable all year round.
and
The level of training at the peak may be too demanding and again not be sustainable all year round.
On the other hand, finding the highest level that is sustainable and then picking it up a little more for key races, rather than constantly up and down. I think this is what Joel Filliol works to.
I’d say that it’s something that most age groupers do rather than taking an off season then building back up again. Maybe the pros do it that way as they have the time?