I’ve never been challenged by walkers for riding the Pennine Way from the A62 to the A640, it’s part of one of my regular loops. Good fun, with a few interesting features to keep you on your toes. I don’t know anyone who’s cleaned it dab free. The bit north of the A640 to Windy Hill is good too and I’ve not had a problem on there either.
I do stay away from the PW on sunny weekends and bank holidays, though, when you’re very likely to encounter huge herds of red-socked OAPs and other assorted gormless grumps. There are a few completely non-cheeky alternatives on the Diggle/Delph side of the hill, look for the Pennine Bridleway signs, you can ride legally almost entirely off-road from the A62 up to the A640 and beyond via the tracks round Castleshaw and Dowry reservoirs (marked as footpaths still, but actually part of the Pennine Bridleway so completely legal AFAIK).
The Packhorse Road down to Marsden, which I assume is the bridleway you are talking about, is a fantastic descent. It is currently in brilliant condition, the boggy middle section is almost completely dry (or it was a couple of days ago, last night’s torrential thunderstorm might have changed that). It is a bit technical in places – I had a couple of entertaining offs on there on Monday. But as a rule you end up landing on nice soft grassy tussocks so it’s really not too bad even if you do screw up.