I used to go out MTBing with a group local to me – it was handy for finding routes new to me and most of them were thoroughly nice people.
But it was very definitely pootle pace. Stops at the top and bottom of the hill to regroup, quite often a longish lunch stop. It was just a mental thing for me to come to terms with and once I accepted that I was in for a slow and easy day out, it was fine.
Accept it for what it is. Bit of a social spin that may even pay off in terms of your job and you can use it as a recovery ride for you and maybe even help them out with some motivation, encouragement, advice etc.
I’ve done ride-leading/chaperone work on corporate sportive/challenge style events in the past and it’s full of people like that. There’s no sport to be had in out-riding people like that unless they’re being monumental cocks in which case the rules go out the window!