If you were upset by it then it was too far.
if you had a quiet word and he carried on then your “mate” sounds like a bit of a dick.
I know a thing or two about ripping the piss out of people but I have a few rules….
Never have malicious intent
Make yourself the object of ridicule as much as you do other people
if anyone objects or takes offence,stop and apologise
Don’t dish it out if you can’t take it.
For me, these are what constitute banter. Anything beyond that is verging into bullying.
edit: unless I come across someone like your mate, in which case , all bets are off and I’ll open a righteous can of sarcastic whup-ass upon them.