Hmm.
Contract of sale: They have offered you goods (it used to be called “an invitation to treat,” dunno if it still is), you have shown an intention to purchase, they have given you a price, you have paid that price, they have taken the payment which implicitly means that they have accepted the sale at that price. It’s their mistake and they are now legally bound to honour that contract of sale.
It’s a pricing error, end of. WHY it’s a pricing error is an irrelevance to you as far as I can see. Should they subsequently have to pay VAT on the sale, they’ll be paying VAT on the selling price, not the full value of what it should have been.
(As I understand it, anyway. Usual disclaimers apply.)