I had a TZ10, a TZ7, and before that a TZ4. Neither had problems functioning even with mostly carefree and regular outdoor use. The problem with (I suppose all?) compact zooms is they can suck in airborne particles so need to be in a good case most of the time.
The only compact zoom that has died on me owing to zoom/lens error was a Sony RX100. I sold it for parts and bagged a (used) Pentax MX-1 which is stellar in most situations and feels tough as you like – but I’m now careful to wipe the lens tubes/assembly before switching off/retracting and refuse to get the camera out on a windy beach.
I don’t know of a ‘tough’ compact zoom that has particularly good low-light capablities, either in your budget or above. What about another Lumix and a decent tough case? Or a used camera such as the LX3 as mentioned? Another good used (though becoming rare) bargain is the Olympus XZ-2.