Leave as much as you can at work. It does depend on your job of course and what you need to wear for it. I leave everything at work then take in t-shirt and underwear each day – I’m not in a customer facing role and only have to be “presentable”. There’s no shower at work so ride in steadily trying not to sweat (fortunately it’s mostly downhill) wait till I’ve cooled down, usually that takes about a cup of coffee, and use baby wipes to clean up.
I’ve an Ortlieb 4 litre saddlebag which is plenty big enough for stuff and attaches/detaches really quickly. Proper full length mudguards on the bike keep most of the crap at bay. At this time of year overshoes and leg warmers deal with the rest.
If you want a bigger pack then look at the Carradice SQR bags, there’s a “slim” version which is wider but shallower than the normal one, wide enough to get a laptop in, which should be fine for getting ironed shirts to work.