I swapped for pretty much the same reasons as OP. New wheels and new tyres. They were a faff to setup, partly I suspect because I didn’t clean the inside of the tyres first (Conti X-Kings).
For me the combination of more comfort and grip with no loss of rolling resistance makes me sure I won’t be going back to tubes.
But everyone has different opinions/requirements. So I’d say give a go, if you don’t like it you can always go back.