Aim for 3 months at least in the company bank to cover being on the bench while still paying self a regular salary. Will take a while to build up funds to this. Plus always have enough to cover corp tax.
May or may not benefit from being VAT registered. Some clients prefer or require VAT registered companies. Small enough turnover you can use flat rate VAT and might benefit from that, though less so these days financially, but it still is a benefit from being less hassle.
Check out IPSE, maybe consider Plus membership as it gives you various insurances on the contract side, clients going bust, illness, jury service & tax investigation cover.
Decent freelance specialist accountant and they might have handy guides.
And depending on the work, you may need to get Professional Indemnity and Public/Employers Liability insurances. Prof indemnity maybe the key one in case your company gets sued by the client if things screw up. Liability stuff covers you causing damage/injury etc, e.g. you burn the client’s office down by accident. Clients may require this.
Personally, in the IT world, I’d try to get direct contracts and use your own IR35 friendly contracts (IPSE and others have templates). Use contacts, friends, former employers or colleagues. But if you’re stuck then agents it will be, but they are looking after themselves first and some are really terrible. I find there’s a fight to get a decent contract on your terms going that route.