I'm really impressed by the 150g/m2 icebreaker fabric. I have a long sleve and short sleve version and they are quite tough for such a thin fabric. Not very fast to dry but they are a lot warmer than synthetics when damp.
By far the best 'general riding' combo I have found so far is a skins compression top under either a 150g/m2 (or if cooler weather a 200g/m2) merino LS top. A very warm and comfy set up. wind or waterprrof shell over the top as required.
As for the baggyness, the wool will get wetted out from your sweat then will cool due to being out of contact with your skin. whether it will be any less comfy than a loose flappy synthetic I'm not sure.