If you were to put so much oil in that you half the amount of void space in the lowers, you would double the air pressure behind the seals and perhaps pop them out. A few ml is a small fraction of the void (I'm avoiding calling it air space to avoid confusion with an air spring, eben though that's how it will act) will make little difference.
It's a bit like worrying about shock pressures when you take your bike on a plane. The theory is there but actually, the chance is small and it doesn't really amount to anything.
I assume we're talking about the sloshing about near the bushes oil and not what's used for the damping circuit by the way, you did say in the lowers.