Hmmm... sounds vaguely familiar to how I was prior to getting in the condition I am.
I used to find I'd be tired all the time, wanting to sleep halfway through the day but would fight on through it.
I'd ride and find that recovery took longer than 24 hours and the more I rode the less fit I'd seem to get. I'd also seem to be ill (cold / sore throat) after every ride. I went and saw the doctor as I felt something wasn't right - they said I was fine - blood test OK and that it was just a bug going round.
Over 12 months this gradually but hardly noticeably got worse until I got hit by a bad virus and spent the weekend riding GT - whoops! Never been the same since and that was 7 months ago. In fact I've not ridden my bike since.
For 4 months I didn't know what was going on - seemed to have OK days and then bad days where I could barely do anything. Anything physical or mental would wear me out and leave me washed out for a number of days afterwards. I'd rest for a week or so, then think I was fine and relapse quite severely.
Anyway, to cut a long story short I was diagnosed by the NHS as having ME / CFS / PVF and signed off work for 2 months - bit of a catch all for when someone doesn't recover and they can't pin the tail on the donkey. They basically told me to rest and I might recover if I was lucky.
Not really wanting to believe I had no control over my fate, I went off to find out what I could and over the space of 2 weeks I managed to learn a lot about what was happening to me at a physiological level. After seeking some help privately and having some tests carried out, it turns out my adrenal glands are not functioning properly and need to be nurtured back to full health.
Apparently Adrenal Fatigue is very common at present and more and more people are exhibiting symptoms these days, although the NHS only recognise Adrenal Failure. Do a search on it and you'll see what I mean.
Since changing my diet, taking regular rest, introducing supplements and various other lifestyle changes I am slowly getting stronger and have more positive days than bad - no more peaks and troughs but I can't ride just yet but that day is not far from my grasp.
All I'd say is listen to your body, if you don't feel right then you probably ain't right. I learnt the hard way, but had this not happened I would never have learnt what I have and I have made some really positive changes for the future.
It may be that this is not something you need to worry about but I wanted to mention it so you can check it out and see if there is anything in it concerning your health.