Overfilling is totally possible, and frequently recommended by people who don’t understand why it might be a bad plan.
It’s what happens when people do a bleed with part warn pads or advance the pistons while bleeding. It used to blow the lever seals in Formulas all the time.
You definitely want the brake cleaner out ASAP and the system is going to need a good old venting followed by a lot of sacrificial clean brake fluid now. It’s possible that the ‘gunk’ was actually from internal seals, so once you get the system back to pressure assume nothing and seriously consider testing extensively in a safe environment where sudden brake failure would be OK before using them anywhere that you don’t want to possibly discover you no longer have brakes.
The only thing you should ever be pumping through a brake circuit is the fluid it was designed to use. Brake cleaner is strictly for exterior surfaces as it’s vicious stuff.