You could try a switch at both ends, and use an HDMI over IP box.
vLAN separation if you want to be clever but likely not necessary.
HDMI to IP convertors work OK though I’ve not tried them for 4K high movement feeds – might be a question of how much you are prepared to put up with. I think to ge 4K you are looking at pro AV kit, HD you can source on Amazon / EBay – but it has to be over IP, not just a line extender / using the CAT cabling standard (they often call that Ethernet which isn’t strictly true but I’m sure you get the idea).
If you could re run a 6A cable you could get 10Gbps to link, more than enough, but the costs go up. I don’t think you need to match HDMI full bandwidth as the convertors reduce that for you, but I would expect a quality loss as there usually is – YMMV. 5e is good for 1Gbps anyway.
You ought to check your XBox controller works through the wall too 😀