There will be comprehensive records of why the building was listed, and it's construction. Also, listed buildings are gradually being photographed along with the 'list description' and added to the English Heritage website. So, as above it will have to be reinstated, also fines and possible imprisonment is possible.
I watched the Beeney program, and thought the owners of the listed house by the road were idiots. Buying a listed building, and trying to remove the features it was listed for – people would want to buy it for. They didn't give a fig about the building, just profit. That's what listed building protects against.
I live in a listed buildingm by a busy road. If it didn't have the features it was listed for, I wouldn't have bought it, and lived in an ordinary house in a quiet location.