I have a 04 TDI sport, cracking car all round. I find it very comfortable to drive, although the stock shock's & low profile rims make the sport a very harsh ride on most UK roads. A very capable bike lugger too, as has been said above with the rear seat out the load space is tardis like. There's a few things to look out for, dependng on the model, and there lots of info on the forum mentioned above.
Economy wise, ive got 75mpg @ 60-70mph before. Tax is laughably low, insure can be a bit more as bodywork repairs need to be carried out by Audi approved body shops only (of which there are about 8 or so in the country). General maintenance is as easy/hard as any other car to work on and you don't need special tools to get into the engine bay (except your fingers) The bonnet just comes off, rather than hinges.
Pretty much agree with stumpy01, they were a bit before their time. The economy figures for the TDI would make them a bit more popular today i think. There was a 1.2 TDI version, very much biased to fuel efficiency and they regulary get 100+ MPG. It was never released in the UK though, but there's the odd left hand drive floating around.
If you do go for the TDI 75BHP, its worth getting it chipped buy the likes of Stealth tuning (details on the A2OC forum), if only for the improvement in drivability. The figure were 75bhp & 140ft/lb torque, now 105bhp & 210ft/lb. Obvioulsy the TDI 90 is pokier as standard, but you pay a premium for the 90.