Surely you only allow port 3389 connectivity to IP's on the VPN lan to the TS server?
That way you know that providing that so long as you're using SSL VPN to get to it, you're sorted, and is pretty much what most people do.
However, if you want to tighten up the authentication to the RDS server you have three choices:
"You can enhance the security of RD Session Host sessions by using Secure Sockets Layer (SSL) Transport Layer Security (TLS 1.0) for server authentication and to encrypt RD Session Host communications. The RD Session Host server and the client computer must be correctly configured for TLS to provide enhanced security.
The three available security layers are:
- SSL (TLS 1.0) SSL (TLS 1.0) will be used for server authentication and for encrypting all data transferred between the server and the client.
- Negotiate The most secure layer that is supported by the client will be used. If supported, SSL (TLS 1.0) will be used. If the client does not support SSL (TLS 1.0), the RDP Security Layer will be used. This is the default setting.
- RDP Security Layer Communication between the server and the client will use native RDP encryption. If you select RDP Security Layer, you cannot use Network Level Authentication."
Read here: http://technet.microsoft.com/en-us/magazine/ff458357.aspx
Personally, I'd be happy with two form factor SSL VPN access only for users whom need to get to the RDS servers.
ps might be worth telling us what infrastructure you're using (i.e. W2k8, etc).