It’s all about good beans, correctly roasted and then freshly ground.
Good French coffee IME is filter coffee – possibly with hot milk (the classic cafe au lait). You don’t need a machine for that – any cafetiere is fine. Just heat the milk in a pan. Any halfway decent (burr) grinder will do as you’re after a relatively coarse grind (Dualit is good vfm). You just need to find a source of beans that you like. It’s not going to be from the supermarket though.
Espresso is an italian thing. Cheap and simple is a stovetop Moka pot, (in conjunction with the same grinder above), but for “proper” espresso (and hence cappucino, cafe latte etc) you need a more specialist grinder that can grind much more finely (I have a second hand Mazzer) and then something like a Gaggia Classic or Rancilio Silvia are the default good “starter” espresso machines. You should be able to find them second hand and/or reconditioned for about the £100 mark. The machine is the least important part of the equation though – beans, grinder and technique are what makes the difference.
(oh and pod machines all seem to produce shite. Nespresso was terrible IMO)