It's not quite a single bolt holding it all.
The big countersunk M10 bolt is taking the vertical load. It was only that small M5 bolt that was preventing the chain tension overcoming the clamping force of the M10 bolt and turning the drop out.
Neither bolt is an interference fit, so all that was happening was that all the slack was being taken up in one direction. Although, if the bolt ever sheared, it would have moved a lot further.
I've modified it now.
We've been for a 15km ride and nothings moved, so I think that's the solution.
I did think about just using a bolt as a stop as that would have saved a trip in to work on a Sunday to use the welder, but it's bearing against an aluminium frame, so I wanted to spread the load.