I'd recommend checking out a local club. Whereabouts are you based?
Most clubs will let you have a few taster sessions for free, they will usually be able to lend you all the kit until you know what you're after, and there are always loads of folk only too happy to help you out and guide you.
Good places for families usually too. I started out in clubs but drifted off when I started to go on harder trips. Its taken me to some amazing places around the world and its a fantastic thing to get into.
Sadly, mostly due to convenience, my biking has totally overtaken it now, but hopefully I'll return to it as I left it.
Just as an aside, a BCU (British Canoe Union) license certainly does not sadly mean you can access a lot of waterways. The access situation in England and Wales is appallingly limited and full of conflict with competing interests (Fisher folk mainly).
Anyway, seek out a local club and fill your boots...