I grew up in Combe Martin and went to school in Ilfracombe. There is a mountain bike group that rides from the pack o' cards (Combe Martin) at 6.30 every wednesday night (all welcome, no one gets dropped etc) and plenty of riders around. Southfork is also an ace bike shop and have club rides
As mentioned above, Ilfracombe is probably the worst town in North Devon. It is surrounded by lots of lovely places, and has its plus points, but I could not (at all!) recommend you move there!
As for Combe Martin, its only 10 minutes away but its a whole 'nother world. I can ride up onto exmoor from my doorstep, wheel the kayak 3 minutes to the beach and explore the coast, and its 30 minutes in either direction to Lynton or Croyde for surf no matter what the tides are doing.
North Devon is fantastic!!