Calorific content is measured by burning and measuring the heat produced. By that measure you could get fat eating a chunk of coal.
What it doesn’t take into account is that some parts of food are indigestible to us so we can’t use the energy /turn them into fat. Also that some foods take more energy to digest.
So I reckon that when you eat processed foods you use a higher proportion of the calories that they contain compared to whole food where you have expend energy digesting them plus you won’t get the full calorific value out of fibrous / part digested parts.
I’m with your wife, Kryton.