5mm will probably be ok but i don’t think you have got your maths quite right.
Current set up is 185mm steerer with 5mm spacer. Say you have a 2mm to acomodate the top cap going to 175 would give you 7mm from the top of your stem. With the open design of the Thomson x4 you may find this pushes the steerer below the top bolt to the unsuported area of the stem.
I may have miss understood what you have typed but to my understanding it is not going to be 5mm from the top of the stem which would probably be ok but 5mm less than your curent set up which may make it 7 or 8mm below the top of your stem which is pushing it.
[edit] damn you craig beaten to the post and he even sticks a picture in [/edit]
* wanders off chanting must type faster must type faster