Heat transfer is not the reason, think about it, I presume the cold is off the mains, (very small possibility it is tank fed unless it's a shower or bidet, you will know by the pressure) how could it have enough time to heat up from the point it enters the house to the tap, initially yes but once running for a few moments no. I am presuming you live in a house, you may live in a block of flats which may well affect it.
The same would apply to the hot.
If the cold is tank fed it probably would be very slightly warmer than mains.
What heats the the hot? adjust it or is it faulty?
I notice a difference in the cold water temperature from summer to winter, my electric shower has to be set differently to accommodate the different incoming water temperature.