I’d say a 29er hardtail with fast tyres on – mine is almost as fast as my commuter except in a strong headwind.
You will have to spend significant ££ to get really fast tyres, and go tubeless, so it depends on the faff you can be bothered with.
A gravel bike will be very marginally faster on your commute, but very compromised if you want to do anything tricky off road. And for the majority of the off road a gravel bike does work for, the HT will do it a bit better in a bit more comfort.
Of course the correct answer is buy both. And budget for another few bikes 🙂