Bank holiday weekend so it will be pretty damned busy, traffic could be very bad depending on what direction you come in from. Town is nice enough though.
If youre around for the weekend come up to Staithes/Runswick Bay/Sandsend for a look around, will be busy-ish but still fairly quiet.
RHB is worth a look but be prepared for a damned long walk down to the bottom of the village down a really steep hill! Worth it though, nice cafes at the bottom and more second hand bookshops than you can imagine.
Inland there’s the little villages such as Grosmont, Castleton and Danby. Danby has the Moors Center and some lovely walks, theres also the Stonehouse Bakery there (was a frequent place to grab a good cuppa and a sandwich for a number of years for me).
Have a look at Whitby Today website to see what events are going on over the weekend too.