Don’t know about NZ, but I have some experience in OZ desert riding and North Queensland.
Water and more water. In the north you can be dead very quickly without water. If you start feeling lethargic or stupid, drink.
It depends how remote you’re going. Base your water needs on what would get you back to civilisation if you were walking. Always keep your water topped up when you can.
The bike – the simpler the better. Tubeless or plenty spare tube and patches if you’re venturing into an area with spiky stuff. Chainbreaker and spare derailleur hanger as well as the usual spares you may carry on a tour.
And don’t be the typical Pom who tries to get a suntan and ends up in hospital or miserably burnt – cover up, including hat.
Presumably you’ll be taking a compass etc.
Down south it’s much gentler, but it can still catch you out.
Edit: and don’t worry about snakes etc – avoid if possible, but most will get out of your way. Get out of the way of those that don’t! Also if in the north, don’t camp within 100m of a creek or remote beach (predatory water geckos).