All other things aside, which job do you want the most? Take that one.
Loyalty to a new company you’ve not even started at is laudable, but misguided. Companies routinely lay off employees who’ve worked for them for decades. If you renege on company B they’ll go “oh, that’s a bugger… who was second on the list of candidates, then?” They’ll be cross at you for about ten minutes, a week later they won’t even remember your name.
I was in a similar situation years ago when I was contracting. I’d not been contracting long, lined up a short-term job for a couple of weeks. Then a couple of days later something awesome came up. I knocked it back, figured it wouldn’t reflect well on my career to back out of something I’d agreed to. I’ve regretted it ever since.