Banff is a dump in the same way that Whistler, Canmore, Jasper etc etc are all dumps. You don't to any of them to look at Norman churches, thatched roofs or castles. Aesthetically speaking they are all pretty boring (I think I spent around 30 minutes in Banff itself, but I don't spent any more time in Whistler than I have to unless I have a bike between my legs...), but it's what's around these places is what makes them great. Just stop in the towns for a coffee and cut some bum rope. Another view near Banff:
As already mentioned, the Icefields Parkway is a total must. It's also not in between Calgary and Vancouver but whatever, you can just drive up from Banff, stop at Athabasca and turn around and head back down again and it'd still be worth it. It was mentioned that Drumheller is crap but there's something to be said for being in a desert, and then a few hours later standing on a glacier.
Crownest Highway between Calgary and Hope (and then on to Vancouver) is a pleasant drive. You can also head up to the Okanagen from Osoyoos (don't stop in Osoyoos unless you want to mix with a million dicks from the Vancouver burbs....) and check out the vinyards.
As long as you don't spend too much time in Calgary or Langley I don't think you can go too far wrong!