Deeside way (old railway line) is pretty obvious on OS Maps. I know you said no roads but the best option I can think would be Aboyne to Ballater on the deeside way, then cross the river in Ballater and ride the wee B-road road (much quieter than the A93) to Balmoral, cut through the balmoral estate (access depending) to Invercauld bridge (cross on Bridge of Dee). Ride through Keiloch carpark, Invercauld house and continue on track along North side of Dee (turns into an unclassified road at Quoich) all the way too Chest of Dee. I wouldn’t want to go any further than that with a 3YO as as daveatextremists says it’s a rock strewn single track path.
The worst bit would be the 8 miles from ballater to balmoral on the road, but it’s a pretty quiet stretch. I’m assuming you’re talking about trailer or seat?