It comes down to how the job was quoted/estimated (there is a difference between the two btw, but that’s another discussion).
I may estimate a job in two different ways for a client, thus:
Say the job was 30sqm of Oak flooring, I may say:
To supply 34sqm Oak @ £x per sqm
To fit & finish oak @ £y per sqm
In which case, if I’m efficient, don’t waste too much, and there’s a few sqm left, it’s the client’s to do with what he or she wants.
If I say:
To supply, fit and finish 30sqm Oak @ £z per sqm, then I order what I think I need and what’s leftover at the end is mine to take away.
So, really, it depends on how the job was estimated. If he’s purposely over-ordered, charged you for materials, knowing there’d be some left over for his private job, then he’s done a naughty IMO. If you accepted his price to do the job supply & fit, then the stuff is his (again IMO).
This is why I encourage any friends and acquaintances to get a reasonable breakdown of costs in the first place. The stuff I’ve seen disappearing off site when a client’s not paying attention…