If you get something burly for downhill, will it not be heavy for cross-country?
I of course strongly recommend having one bike for each different type of riding 😉
OK, I'm the same height as you and ride an old style Juliana. I believe the new ones can run with a 120mm fork. My lbs have a demo Juliana which would fit me and it's a small.
But obviously women and men come in all shapes and sizes, for example, my other bikes are all men's!
Another example, when I was looking at Scott road bikes, the men's and women's geometry was identical. The only difference was the short brake levers.
I would suggest asking the guys on here if you could try their bikes for size cos not all of them are tall.
Hope this helps but it's a nightmare!