I'm not a spokesman for urbex, there's no "urbex community", everyone makes their own decisions.
However "wrong" is a difficult thing - that's the problem with all moral decisions. I'm not a fan of street photography, I think it's rude and intrusive to take pictures of people who haven't given permission, but I understand that it's a valid form of the photographic art.
There are many examples of street photographs being used without the subject's permission.
Yes, some trespass is illegal - I broke laws by exploring Bishopton and was charged by the police. That's why I said "usually" - most explorers don't do that. Though police do often try other things - the catch-all offence of Breach of the Peace is often used.
If it was all safe and legal and sensible, it wouldn't be fun