Alright pay close attention. I’ve had exactly the same problem as you before and it was driving me nuts.
You’re right about the preload bolt. It’s only purpose is to preload the crank so it can be tight enough before you do the main arm bolts. It does NOT hold the crank on by any means.
The fact that your crank arm gets undone means that it does not have enough space to properly hold itself on the shaft.
Let me take a wild guess here and say you probably have one extra spacer on there. If you have a 73mm bb you only need ONE spacer on the drive side. If you have a 68mm (or whatever the smaller one is) BB then you need TWO spacers. One on each side.
In my case, the dude that put the bike together had made a mistake and used TWO spacers on a 73mm BB. That resulted in the crank arm being pushed ever so slightly outwards, enough for it to not maintain grip properly.
Check the length of your BB and how many spacers you have on. If you see a mistake, fix it and then do everything up again using the new preload bolt that you ordered. The crank should not fall out any more.
Hopefully there is no damage done to the grooves of the arm and you can just do everything up normally and ride on!