I do alot of this, bivvy bagging and guerrilla camping with a small tent. Got the bug years ago when doing DofE, as part of it we had to spend one night in an old orange plastic "survival bag" but my group only had a small tent so I ended up doing the whole week sleeping in the plastic bag! Bivvying is great and as others have said you can easily get your overnight kit down to an easy weight. Always use a 30l backpack including water bladder myself and if the chest and waist belts are done up well I find it fine to ride with. If you're out with your bivvy bag my top tip is look at the weather forecast! Not the general one on telly after the news but learn to read the pressure and wind charts on the met office website, with a bit of experience you can guage pretty well what the next day or so has in store, especially this time of year. As others have said it's not as great in cr@p weather no matter how hi-tech your bag is.
I'm on a limited budget so most of my stuff is cheap and cheerful I've had alot of use out of a >£20 pro action 1 man tent I got from Argos a couple of years ago, don't know if they still do them but a friend got one off ebay earlier in the year. Surprisingly robust, not too bad weight-wise (about 1kg-same as a litre of water), give it a good fab-sil and its waterproof and packs up small especially if you put the poles on the outside of your bag/tape them to your frame, leave the door, essentially one side of the tent open and it's pretty much the same as being outside with the benefit of a bit of shelter if the weather does turn. It has a net inner door aswell for insect protection. Only one skin so does condensate badly (especially hassle in sub zero temps when the condensation freezes-was a lovely clear winter night though!)
Agree with the other top tips, wooly hat, fresh socks etc. mine would be long johns to change into. Sleeping in padded cycling shorts is not good. Also a sleeping bag liner, adds a bit of warmth if you need it and easy to wash.
TJ-You can change in a bivvy bag, just takes a bit of wriggling.
Doughnuthead-I always have my mat inside the bivvy bag, means you don't slide off it in the night.
Sorry about the essay, after mountain biking this is my favourite subject