Swimming is all well and good as long as you don't get cramp or hypothermia, or have a heart attack or something. I believe that's where most of the problems occur in still inland water.
I think part of why I'm very confident in flat water, is that I know that I can swim a km with legs or arms only, so even with cramp, I'd get out.
Hypothermia is a pain, and you do have to avoid it, but to be honest you'd be surprised at how long you can swim in pretty cold water without getting it, as long as you keep swimming. You do have to be slightly careful, and pay attention to your body. And also be aware that the really high risk point is when you get out into the cold and wind and stop swimming, at which point it is a good idea to get into any dry clothes you have handy and do a little dance until you're warm again!
The only time I've had any cold water worries, was filming this youtube vid – because it was white water, there were a fair few pauses to stand on rocks and check the line on the next section, or just to get my breath back, so I did end up getting a bit chilly, and having to get out on a little beach and jog on the spot for 30 seconds, half way along. That river was dam release coming from the bottom of a big lake in the mountains, so it was jolly cold, although the weather outside was 25-30 degrees so it didn't matter much.
Joe