From Brizzle, get to the old Severn Bridge, then follow the lanes up the English side (basically, take whichever is the nearest one to the river that’s not a dead end, it’s pretty obvious from the map). You’ll go through Berkeley at some point, then to Purton, where you pick up the Sharpness canal all the way to Gloucester docks. Work your way through the centre of Gloucester (literally past the cathedral, past Kingsholm, then onto the A38 up through Longford and Twigworth. Turn left at Norton and drop to the river (Red Lion at the bottom of the hill by the river might be a good lunch stop). Cross the river at Tirley, then hang a right and follow tiny lanes skirting the other bank of the river to Forthampton. This is stuff I know well and is pretty much all pretty lovely.
From there, looks like you pick up the B2411 and stay on it all the way to Gt Malvern, but I’m not as familiar with that end.