I had the same problem and wanted to leave early so I requested that we discuss it. My old employer ignored the issue till I pointed out that I didn’t need a reference for my new role and would be leaving on receipt of my next payment. We then got together and agreed (I told) what date I would actually finish. If they’d been open to communicating from the start I would probbaly have stayed longer as I would have been able to handle the situation with my new employer better.
The gist of it is that they can only come after you for the financial damage you cause by not working to your contract. The cost of doing this compared to the actual damage they would be able to prove means that they are extremely unlikely to.
In my opinion you’re better off not burning your bridges and leaving on the best possible terms. You never know when your paths will cross again with your former colleagues if you work in the same in the same industry or area.