Passats and A4s have a dull as ditchwater reputation for driving experience, but they're the best Derv engines of that era really.
Two problems – they hang onto their money like crazy. Compare the prices for a VAG diesel to the equivalent Mondeo TDCi or Pug HDI and you'll see what I mean. The other thing to look for is history, just so you know it hasn't been clocked. They can stay very tidy looking for a long time.
The A4s and Passats both share the same design under the skin. The front suspension wears, and there are lots of arms to replace to get it all back to good nick again. Can be very expensive.
The A6 is obstensibly on a different platform, but I wouldn't be surprised by major similarities in suspension & chassis design etc.
Injection pumps on the old diesels can get horribly noisy.
I'd personally be inclined to go for a 1.8T, which will have plenty of go and reasonable economy. I'm sure they'll also be a lot cheaper to buy than the diesels, which may work out better if you don't do major miles.