“activity aimed at improving physical fitness and well-being, forming social relations and gaining results in competition”.
is Sport England’s definition.
The problem with that is whether there is such a thing as mental exercise, and whether it can diminish the chances or delay the onset of dementia, which is obviously a major public health problem in an ageing population.
http://www.bbc.co.uk/news/health-23159127
I suppose they have to draw the line somewhere, but the difference between sports, skills and pastimes is not quite as clear-cut as you make out, especially when you have sports such as darts and target shooting which do not arguably improve your general physical health.