Rental or buy? If rental, then you need to time things well when you move out otherwise you can end up leaving a few hundred quid if fuel behind.
I reckon it is more expensive than gas but cheaper than electric. It is cheaper if you have a big tank and can stock up in summer, prices go up during cold winters. Many areas have local coops which get better deals. Also bear in mind you will always be prepaying.
Also, both places I have lived with it always had a faint whiff of diesel around the boiler, which was less of a problem in the place where the boiler was in a separate utility room.
I wouldn’t rule somewhere out because of it, and in many rural areas it’s inevitable unless you go for gas cylinders which I imagine are more expensive and hassle.
Delivery is not a hassle, you usually pay the supplier over the phone / internet and someone turns up during the day (you don’t need to be in) and fills an outside tank.