You didn’t mention the wall thickness, but I think that this is probably possible. However….
You would need to do this with something mildly abrasive and uniformly to avoid having some areas thicker than others. A lathe with a feed and an attachment to hold some emery cloth for example, I would also lubricate with water.
The average UD cured ply thickness is about0.2 so you are not losing a great deal of material. Also as someone mentioned a proportion of this would be lacquer. I don’t worry about UV deterioration as most epoxies are resistant these days.
I would advise against the use of normal turning tools as they can promote delamination and lifting of fibres. This is not the same type of removal that we are after here.
As someone mentioned, reaming out the seat tube is also an option and possibly easier depending on what tool you have available.
Good luck!