I too have considered a change of career but on balance I take more positives out of my work than negatives. though sometimes it is a close split! its a roller coaster of good times and hard times.
Is there really nothing you can focus on with your work to help cancel out the negatives as often these things are just a frame of mind? No one has the perfect job. Otherwise have you considered staying in the industry you’re in but doing something different, i.e become a personal home tutor / coach or if you currently work in schools, perhaps look at teaching adults or going in to college education?