I think it’s quite a complex answer. On the one hand how many classical composers have stood the test of time? Considering the amount of time involved not that many. We only really hear who are/were considered the absolute best.
Modern music has been all about capturing the zeitgeist. Our society has become more and more obsessed about staying ‘young’ stemming from the post war period and modern music has reflected this. How can an artist in their middle age continually speak meaningfully with a much younger audience under this criteria.
There are a lot artists who write continually strong pieces of music but they are generally outside the mainstream and so not many people get to hear them. Others, as has already been said, stick to the formula that made them famous in the first place as at the end of the day they have to earn money to live.
Lastly don’t forget it’s an industry with the artists being the tip of the iceberg. Bands for example are often coerced into writing music that they don’t really want to as record companies want to maximise their profits. On the other hand it can also be argued that there are a lot of musicians who write the hits for other artists and have been doing so for a long time but I suppose that is more based on success than artistic merit.