PAYE in your specific case is basically a zero hours contract. There’s no hassle on your part regarding setting up a company (sole trader or ltd). You will be an employee of your ex-employer. You will probably get some entitlement to pro-rata holiday pay. But don’t quote me on that.
Ltd has risks and hassles. The risk is that you end up having to pay income tax on your earnings (at the same rate of tax that you would pay if you were on PAYE). If that happens then you eliminate any benefit of going ltd, and are still stuck with the accountancy and administration hassle of running a ltd company. In your case, the risk of that happening is high, since you will be solely working for an ex-employer. If you are lucky, then you will avoid this risk (IR35) and may end up paying significantly less income tax than PAYE – depending on the income you earn. As I said before, if you earn less than £45k then these tax benefits won’t apply.
To me it sounds like you’ll be earning less than £45k, you’re at higher risk of being caught by IR35, and you don’t like the sounds of the hassle of managing a ltd company.