I used to ride that route as far as Hammersmith Bridge so I can vouch for that section. Googling the second bit of the route, it looks like the route is fine up to around Putney Bridge then goes a little on/off the river for a bit. But you’re pretty close by then so I’d say it’s probably as good an ‘off road’ route as you’re going to find.
Just in case you’ve not been this way before you’ll need to come off the canal at Brentford lock then follow the road to Kew Bridge, crossover the bridge and join the riverside path on the south side of the Thames.