The main reason is grip through better conformance to micro terrain. A softer tyre squidges over small bumps instead of striking them and then having to ride up and over them, which also improves rolling resistance.
I’ve seen my tyre pressures progressively drop over the years. When I started mtbing a mere 16 years ago, I used 40-45 psi, with a bit more experience and wider tyres it dropped to 35psi, but that seemed to be the limit with 2.1 inch, 26er tyres with tubes.
I saw a significant drop to about 24-26 psi once I went tubeless with 2.2 26ers and mid width rims like Stan’s Flows, and I’m now down to 18-20 psi on 2.3 29ers with even wider rims.
I’ve learned that tyre squirming is a function of sidewall stability. You can improve this with wider rims and thicker sidewalls. I’d argue that having both is good practice as part of the package to both resist the squirm and provide adequate pinch flat resistance when running low pressure.