cutting down a mature shrub is a totally different activity to weeding, the clue would come when you need a saw instead of a trowl.
And clearing a garden of bricks, old exhaust pipes, heavy duty bags, and a whole assortment of general rubbish, is also “a totally different activity to weeding”. And yet the gardener appears to have been expected to do that.
I suspect that he was told to generally tidy up the garden, and in the absence of anyone being there to make executive decisions the gardener relied on his own initiative.
The only reason I can imagine why someone would be happy to leave a new gardener, they don’t know, on their first day, unsupervised, is that they weren’t too fussed by the outcome.
Otherwise being there to ensure that the work was carried out in a way which satisfied them would be the supremely logical course of action.