It won't be lactic acid; you get lactic acid build up after shorter (less than an hour) high intensity periods of exercise. A day at work wont cause this.
The problem is more likely to do with vascular return and blood pooling. If you're standing up a lot, or generally dont get a chance rest your legs at work, it will cause more blood to stay in your legs, slowing down the rate at which waste products can be removed, and in turn, how quickly freshly oxygenated blood will be pumped to them.
Although your body will sort this out in the first 20-30 minutes of steady exercise, it will still make you feel slow and terrible for the beginning of the ride.
You should try RICE: rest, ice, compression, elevation. Although more generally applied to injuries, it will help vascular return too. Resting is obvious. Ice will stimulate blood flow to your muscles, bringing oxygen and other essentials. Compression will aid blood flow by preventing blood vessels and muscles from expanding to accommodate excess blood, and therefore causing greater venous return. Elevation will give your veins an easier job of returning de oxygenated blood back to your chest.
Or it could be that you just need to man up and ride more!