The referendum is advisory, not binding.
Now in the absence of anything else the government would be expected to follow it.
However on a very close result things get interesting.
The petition forces a parliamentary debate. MPs generally favour remain.
If I debating a close result they come to the conclusion that on balance, after taking the views of the public into account, and effects on national security and stability into account, break up of the United Kingdom into account etc, etc, an Article 50 move is unwise, then the government can make that call.
No rules broken. Decision made.
If.