Depends what you plan to store on the shelves for how much meat to leave but unless it’s for your extensive collection of lead weights, then I’d be fairly happy with say a 20mm batten. I would site the batten off centre towards the bottom face to give you more meat on top.
The question of width comes into it. If it’s quite a wide alcove then you may need to consider things differently.
Having a batten at the back routed in will help support things. With that thickness of Oak, it’s the weight of the shelf you need to consider along with what’s going on it.
If it’s not obvious, do a stopped groove in the shelf so you don’t see the batten from the front.
There isn’t inherently anything wrong with how you plan to do it but, you are perhaps unlikely to get a nice, super-tight fit. Reason being that there isn’t an alcove in existence that’s square!
If your alcove is wider at the front than the back for example, you’ll get a gap as you push it in.
There are simple ways of templating the exact shape, but if you have the scenario above, then you physically won’t be able to slide it in.
If you’re going for a more chunky, rustic look then perhaps a gap here and there won’t bother you.