Not sure I agree with the figures used by somewhatslightlydazed – but agree with the principle.
Riding in a bunch, there is no noticeable benefit from aero wheels, the only benefit will be when you hit the front of the pack for your pull.
My 65mm carbon wheels are lighter than most aluminium wheels and my 40mm disc brake wheels are not much heavier.
But the principle of aero benefit on a gravel ride, unless you are riding very flat smooth gravel then there will be little benefit. And as I mentioned in my first post, depending on the rim, you may also notice a reduction in performance, but that will also be as minimal as any performance advantage would be at 16 or 17mph.
Personally, I’d say stick to what look the best!
In terms of the frame, aero wheels do not need an aero frame to get aero benefits. The frame makes up a surprisingly small amount of aero difference, unless you count your body position. But the aero wheels will give a performance benefit. The faster you go the more benefit you get. (but that’s a very general principle, and at the speeds you will likely be travelling, we are talking single digit watt differences)