Your gut reaction is right. It can be rough.
Lived in Heathfield approx 3 yrs ago. Only lasted 6 months before my wife and I couldn,t take any more and had to move on. Coming originally from Belfast, you would have thought I would have been used to "interesting" characters but Heathfield was something else!
We lived on Lower Cannon Road and spent every night being woken up by our soft-rock loving neighbours. The houses are very poorly constructed with no sound insulation at all.
Houses are cheaper in this area for a reason. There is no local pub as the land was owned by the Mormons and they only permitted houses to be built on it if no public house was to be built.
As a matter of interest, it turns our that some of the kids I am working with (I'm a teacher in a referral unit) also live in Heathfield. They also confirm that there is nothing to do and that lots of anti-social behaviour takes place. Nothing has improved in the past 3 years and word is that it has actually gotten worse.
If you are serious, pop into the Police station in Newton. Have a chat with the PCSO for Heathfield and ask their opinion.
Two stabbings and a murder inside six months cemented our decison and we are now living in Exeter and loving it.
PS. Bovey is nice.