The Fat Cat is very good for beer, and while they don’t do food the cafe/bistro place across the road delivers to the pub. It’s also opposite Norwich’s finest bike shop. Not all that close to the prison, but not much more than 5 minutes by car.
If you want somewhere nearer the Coach and Horses on Thorpe Road is really good on both counts. No bike shop though.
I don’t think you’d need a full susser for Mousehold (where the prison is) but there’s some lovely riding there both sides of the road as you go up the hill from the prison.