Wait until late autumn / winter* and then spray glyphosate (usually sold as "Round-Up" but comes in other names too) onto the stumps. There are other specific chemicals that'll do the job. Drilling holes into the stump will create a bigger surface area for the chemical to get into the tree so that might be an idea.
It'll take much longer than physical removal but a lot easier. The chemical doesn't actually rot the wood, but it does kill it and encourage pathogen infection which will eat away at the stump and roots. It may take a few months and several applications, depending on the size of the main stump and size of the root system, so be patient.
It's best applied to freshly cut wood, so if they've been stumps for a while, take a slice off the top of the stump to expose fresh wood.
* The sap in a tree travels into the roots during the winter months and up from from the roots in the summer months, so by spraying in the winter months, the glyphosate moves down into the roots with the sap.