Hey cheers_drive
I wasn't always a graphic designer, I actually graduated as a software engineer. After 2 years in industry I realised I'd made a mistake. To be frank, I was very good and often wonder what I'd be earning now if I'd stuck with it. But I hated it, and was dreadfully bored and exhausted by the whole thing. I'd always fancied being a graphic designer but my parents steered me in the maths/computing direction.
Spent a few years self teaching and hobbying and eventually started applying for jobs. Been doing it for over 3 years now and much happier.
So it's doable and I'm finally on more now than I was when I quit Software engineering. As for what you could do, maybe look at careers that overlap slightly as that'll make changing easier and quicker. Although I was doing Software engineering, I moved into UI desgin, then web apps UI design, then web design in general and I've expanded into graphic design since.
I will say don't hang around if you're fed up, life's too short as they say!