Interesting read for me this (I'm Head of maths in a secondary school).
It's a typical hard GCSE higher question, there's usually one rock hard question at the end of one of the papers (they do a calc and a non calc paper).
BTW Stoners solution is right, (getoyourbike), you can't solve the resulting quadratic by factorising as it won't factorise (Factors of -5 are either 1 and -5 or 5 and -1, neither of which pair with x and 2x to make -4x.... although I agree that's always the first point of call and what I would have tried to do first.
(Tired), well spotted that it is a circle with straight line of negative gradient passing through it, unfortunately you can't solve it geometrically, you could have done if the line had passed though 0 though as you could use the radius of the circle and two lots of pythag... but as you have realised, it doesn't work as the line passes through 2 on the y axis.
P.S. (Gofaster stripes), the current maths GCSE is already no coursework and usually just final exams (modular system is now nearly phased out with the last exam series in March 2014)
P.P.S The current Maths GCSE works brilliantly, there is no need for Gove to interfere with it, it aint broke!
P.P.P.S (CHB) your program is giving a close approximate solution and the right answer is stoners. It probably plots the graphs and looks at where they cross, but doesn't work on an infinitesimaly small substitution of point goes to the nearest ppoint/pixel or however the program works.. it's out very slightly