Depends, is the correct answer. I’ve got some V2 and wanted 160 on the rear but they didn’t make a suitable rotor.
The other problem was a clash between the caliper and the adaptor. However, if you have a 160 post mount rear, it should bolt straight on. As the V4 brake track is the same depth as all the others now, you should be fine. However, these are my thoughts on it, I’ve not actually tried it.
I don’t see why there should be any pad contact issue, it’s the adaptor that sets the size, calipers are the same mount. No adaptor means no change in size.
Having said that, I’ve gone from V2 braided hose 203/183 to E4 Kevlar hose 180/160 and not noticing any drop in performance.