I can comfortably ride for at least 3 hours offroad – singletrack, steep climbs, farm tracks etc. I also did London to Brighton last year in just over 4 hours riding time (lots of forced stops though) and felt fine.
Same sort of thing as me and then I did the 65 miler in 2007…as mentioned, one of the waterlogged years: I found it very tough and it was a lot different than my expectations mostly due to the shear quantity of climbs. Went back the year after much better prepared plus it was dryer and it seemed fine.
As mentioned, I’d say give the 65 miler ago and then aim for the 100 miler.
iirc I can see there’s someone on this thread who’s done the double (edit: plus I’ve now seen your profile)…hats off to you hazeii. :-)