Hey Nasher, I'm using one for pretty much what you are. I have a panasonic gf1 which is fantastic. We are lucky to often have lots of light eh?! The 20mm lens means it goes in a bag on my pack strap and I don't notice it. Lives there and rides with me every day for the last few years and its been fine. I shot all my videos with that. I have tried a few other similar cameras and the diffences are slight enough to not make a difference to me. Recently I bought a bigger camera for shooting on special occasions, the quality difference is huge and video is amazing but it can't live on the bag strap so is pretty useless for what I use the gf1 for and what I think you are wanting. Also the gf1 has buttons for almost everything so I can use it easily with my gloves on.
So basically I'm rambling but I think any of the compact system cameras will be good for you, get a smaller one and get a lens like the 20mm pancake which is amazing and leaves the camera v small. If I had money I would replace the gf1 with the gx7, or the gx1 which you can get cheaper now. Panasonic always seem to score highly on video quality too.