This kind of happened to me a couple of years ago. I got taken on by a small research company but not as PAYE. So I am a self employed sole trader. I work for them monthly and invoice them monthly. It probably falls under IR35, I pointed this out to them so I now have the freedom of taking on my own work. As long as a little job comes in every now and again it should get around the IR35 problem.
Umbrella companies are good – then you can pay yourself dividends and a minimum wage (less tax). However – I am pretty sure the cost of an accountant a year to do this for me wouldn't cover any tax savings (I may be very wrong though).
If you've been offered holiday that's good. Sounds like they are avoiding PAYE costs their end. It seems to be happening an awful lot nowadays.