Not so – letting all the warm air out of the house doesn't remove ALL the heat from the house, as air doesn't actually hold that much. Most of the heat is in the furniture, walls etc.
I'm aware of the thermal mass of the hard parts of the house, but they convey very little to the heating of the house – i.e. if you open the doors and flush out the cold air, the temp diff from table to air is minimal, so the heat transfer is slow and low, in order to re-heat the air you need the rads on, which needs burning of fuel. Though that doesn't mean you have to re-heat the hardware again (which saves obviously) it's still not ideal. It takes 35 minutes (I checked, last night) just to increase air temp in my flat from 19.5 to 21 degress. 35 minutes of the boiler on full whack and the rads at 65c. To go from ~5C outside temp it would take considerably longer, you can appreciate. To do that daily is nuts. I'll find the article if I can, but killing draughts and air-loss is meant to be just about the most effective method of improving home efficiency, as lost hot air and replaced cold air = comeplete loss.
For plots – an alternative is to buy a complete gopping wreck of a place and bulldoze it. It may be a cheaper/easier option than getting a site with planning
Still need planning, just not quite as difficult as dropping a new house in a previously open green field!