I’m not going to argue with you. We disagree and that’s fine, but my view is that the building operations you propose do extend beyond the original brick elevation.
Because there is another point of view (i.e. the one I’m putting forward) which those who have the ability to serve formal enforcement action against you could legitimately adopt (which then you’d have to deal with) it means that talking to those people to discover where they are on the point is probably a worthwhile exercise. They might not care at all. They might care an awful lot. I don’t know.
You also have no idea whether your house is subject to a planning condition requiring that it be built in the current layout.
Or you could just adopt the view “I’m right and only my interpretation could possibly be correct” and go ahead without talking to anyone. It’s not my house and not my headache if it all goes belly up re: planning or building regs either now or later when it comes to selling the place.
You asked for opinions on a forum and you got some.