There’s no way your shorts are wearing through metal!
I’d say it’s because the edge of your saddle is flexing down and rubbing the clamp – just look at the downward striations, plus the shape of the wear is following the saddle edge.
I’d say it’s caused by your saddle being too far forward on the rails. Further back and the saddle sides are further out where they shouldn’t foul. A Flite is fairly low profile to the rails in that forward position, as is your current saddle apparently, and saddles are designed to flex to various degrees.
If you need that position your choice may come to a higher profile saddle, a post with more layback, or most drastically a correctly sized frame!
Edit: too slow