Had a friend who kept his new bike away from his missus by leaving it in my shed and riding round on his hack to swap whenever he went out. Trouble was the cr*p hit the fan when my wife (now ex) found a new bike in the shed and thought it was mine! New GF said she would put up with the growing bike collection if she could have a new bike too…. Job done…

You need to get it checked out. the likelyhood is that it is just a soft tissue injury and once the inflammation starts to subside you should indeed start gentle mobilsation with stretching and weightbearing excersises whilst continuing with the ice therapy. It does sound like one of the differential diagnoses could be tendonitis, or irritation of the tendon sheath.. possibly the attacment of semimembranosis, or the medial attachment of gastrocnemius, even a mild tear of the medial collateral ligament. Hopefully the medial menisci isnt involved.. any locking of the knee? or sudden pain that stops you in your tracks?

Hate to stir things up but cold therapy, anti inflammatories and painkillers with nothing but rest until you see a specialist.. of which im one (podiatry, lower limb specialist). Years of riding, and too many knee injuires to list, and not even i would advise anything other that rest for what is probably the most unstable joint in the body. Not only that but cyclists are only strong in a straight line, being very prone to twisting/shearing injuries to the knee. Rest up now for a week or two, it could stop you being of your bike for months if you continue to stress an existing injury.