If you’re staying on Pen y Gaer Road you’ll probably just have got used to both Garth hill (and the very steep short bit up from Pen Y Gaer Road up to the Panorama cattle grid) by the time you go 🙂
Locally, it’s pretty much 95% cheeky – a nice ride is the one you mentioned (ODA / llandegla do a weekly ride similar to the ride you mentioned ^^) plus there are lots of old quarries nearby, the best one IMO is the long one between Dinas Bran Castle and Trevor Rocks (map), good for beginners along the old line or some gnarr (TM) on the spoil heaps on the right (photo 1 photo 2)
Another nice loop (short, but I don’t mind doing a combination of loops) is – scrolling east on the map above – the track continuing uphill where Offas Dyke Path joins the Panorama, keep on heading up and head either (a) east on the FP to the right of Eglwyseg Plantation, to eventually end up back down towards the cattle grid or (b) carry on until the end of the plantation and left / downhill to head towards the Castle, along the ridge and back down to where the road bends in on itself.