Finding a way to prevent entry is your best option.
Confronting them will, as has been said, lead to escalation.
Reporting them to the proper authorities will eventually do the same when they find out who shopped them, unless all the residents go together, and then it’s not guaranteed.
Other than that, get yourselves some black ski masks, 12″ of snorkel tube with a 1/2″ stainless bolt screwed into the end, cut the head off and put a wrist strap on it so they can’t take it off you and use it on you. Then visit them in the wee small hours, one at a time. Not that I have ever, or would ever, do or countenance any such thing irrespective of how effective it may/may not be. 😀