Pick up hire car at Vancouver airport and drive to either Tsawassen or Horseshoe Bay ferry terminal. Ferry from Tsawassen can take you to either Victoria or Nanaimo; Horseshoe Bay ferry just goes to Nanaimo iirc.
You will need the car while on the Island unless you’re staying in Victoria. Grab a bite to eat on the ferry, you’re on it for an hour or so.
Butchart Gardens is worth a visit if you’re in Victoria but tbh I wouldn’t bother if you’ve been to Harlow Carr.
Whale watching trips typically from Tofino (west coast) or Campbell River (east coast)
Can’t help re Whistler to Jasper but I’d recommend going via Kelowna Valley if you can, pick up some local wines direct from the winery (plenty of wineries on the Island too, especially around Duncan; Blue Grouse winery is worth a visit)
If you go via Kamloops you’ll drive through Canada’s only desert. Yes, desert. No, I wasn’t expecting that either.
Lake Louise is very touristy, Moraine Lake less so, but both spectacular. Lake Louise Village Inn has an economy section which looks like Stalag Luft III. Not recommended, really not. The hotel on the lake itself looks nice though.
Banff is nice but again, touristy. Watch out for trains though, they take forever to pass through the town. Very very long and they travel about 3mph so you could be stuck at a level crossing for quite some time.
Canmore is nice, less touristy than Banff and close enough to Calgary to use for a base; Calgary itself isn’t especially nice – or horrible either – typical North American city. Traffic east-west on highway one can be a PITA