I've done it a couple of times,
1st year was great, and was really looking forward to doing it again,
2nd year was one of my all time cycling low points, lying exhausted under the side of a boat in the rain, freezing cold / tired / hungry and too exhausted to join the huge queue for food. (some nice mates got me some sausage rolls that were amazing!) And then having to fight / queue / wait to get a space for my bike in the lorry back to london. Maybe I'm not suited to road riding, but I found the worst battle wasn't fitness it was a fatigue / boredom that set in at about 3am,
Like I said the first time was good though, so maybe it was a weather thing?