Manhandling an 18kg single wheeler is probably vastly different to a twin wheeler with 36kg* of moving weight and a massive frontal area though.
Yes of course but the point is it makes it harder in some circumstances, like lifting up steps and stuff. Yes, not easy with panniers but you can reach for the down tube for a short distances with panniers to get over a footbridge or whatever, but it’s a PITA with a trailer. Even a high kerb can be a problem with a trailer, or it was for me even when empty.
TBH my main concern would be the weight of the trailer itself, a lot more than a rack. But of course it’s a lot more reliable and as above easier to actually ride with. So yeah, like most of this stuff it all works you just have to choose what you fancy.