yeah. You can ride one plenty of places, way more than you'd expect. Sure, you get battered around a bit but it's all doable. All depends on the rider at the end of the day.
I'd say you can ride them anywhere you'd ride a mountain bike unless it get *seriously* sketchy, and I'm talking jumps and drops here. personally, i'll ride mine anywhere I ride my mountain bikes. i might take slightly different lines or takes things a bit easier in places but they're an immense amount of fun offroad.
I've said it many times, there's nothing like caning along the road at 20mph+ and then suddenly cutting off down a footpath at warp speed, flicking around down some singletrack, trees flashing past your head, pedal, pedal pedal, fly over the pavement and back on the road, wind it up to 25mph. Brilliant!
And you can climb like stink on a cross bike. See all those MTBer's in the distance, you'll be passing them like you've got an engine. More brilliant. For the truley brave, wait for them to catch you at the top, let them start the next descent and then see if you can catch them. Double brilliant.