Vinyl is still the ultimate, always has been but very expensive as top quality turntables are extremely well engineered and expensive to produce. Plus you need an arm and a cartridge.
However you can get a very decent turntable for little money, look at models from Rega (Planar 2 and 3 have been around a long time and lots of used ones on the market) AR also made a decent deck caled the EB101 but my favourite deck for the money was the Dunlop Systemdek made in Scotland, bought one of these way back in the mid 80’s. I had a 11X model. Of course you will need an amplifier and a pair of speakers, you can pick these up for peanuts on ebay especially speakers. Rotel and NAD make great cheap amplifiers and for speakers choose something from Kef, Mission, Tannoy, B & W.