Cologne is very good, i wouldn’t spend too much time at the one near the cathedral though, it gets very overcrowded and is not as pretty as the ones closer to the river (As mentioned, the one with the ice rink), or the one at the other end of town (Neumarkt). Cologne is our favourite German city aswell, and we will definitely revisit it when we leave Germany. The one at Dusseldorf is also nice, very chilled out and not as busy as other big city markets. Avoid Frankfurt and Stuttgart with kids, they are ridiculously busy. I’ve not been to Aachen’s, but that is also apparently very nice.
Some of the smaller markets may be worth a visit. Trier is nice, and Hameln also (The Pied Piper town), but you’d have to fly to Frankfurt Hahn or Hannover respectively. Don’t be tempted to go to Monchengladbach’s though if you’re around Cologne/Dusseldorf, it’s like the shed section at B&Q.