I spent the whole of last week riding my horse around West Somerset. I spent 90% of my time on very quiet country roads, despite there being miles and miles of available bridleway. Why? Because the Bridleways are still very wet, soggy and easily damaged by horses hooves.
To answer a couple of misconceptions: Horses are not tame and they are not domesticated; they are trained. They are also far more intelligent than most people (riders and nonriders) give them credit for but they react instinctively to perceived threats and danger. Lastly, their feet are very sensitive to different surfaces, and broken stony surfaces are as dangerous to a horse as the are to a bike.
To my horse, a flappy plastic bag is a fun thing to stamp on, to a less educated horse, it is a scary monster that is going to try and bite it in the belly (as a natural small predator would). To my horse a cyclist approaching fast from behind MIGHT be a predator, and so he will shift (fast) until he can clearly see that it is not. To a less well educated horse, a cyclist approaching fast from behind (road, mtb, electric or shopper), is a predator, attacking from behind, and so it will run, fast in an effort to outrun the predator (lions in Africa chasing Zebra, David Attenborough and all that, ring any bells at all?).
Why do I ride? For the same reason I ride my bikes; because it's bl**dy good fun and because, when we choose to, we can run at 20 to 30 mph, jumping over fences, logs and banks two to three feet high, and over ditches three or four feet wide without missing a beat.
As for horses off roading capabilities, I have ridden my horse into and out of quarries, rivers and lakes, that I couldn't even contemplate on a bike and up and down slopes that I could barely manage of foot.
Just remember that America was explored and populated long before cars and roads existed, thanks to horses.