I’ve had a tempestuous relationship with my CX bike. At first I really didn’t get on with it and couldn’t really understand where it fitted between my road bike and MTB. The more upright geometry, higher BB and longer wheelbase felt weird. It went through various builds, but now (with a decent groupset on it and some tweaks to set up) I finally get it… to the extent I’m losing the road bike.
Like you say, it’s great for when I can’t be bothered to drive to the trails with an MTB or don’t have the time for a 50 miler road ride. I also have a spare set of wheels with road tyres, but to be honest I’m not sure how much use they’ll get – I like the flexibilty to hit some dodgy surfaces.
If I was really into road biking and after speed, then I think I’d want something specific, but as it is the CX is great.