So we have 1.125″, 1.125″-1.5″ tapered and 1.5″ fork steerers – am I imagining things or are stems only sized by handlebar diameter (25.4 or 31.8) and length and rise? If so, how do they fit the different steerer tube diameters?

For a 1 1/8th steerer you buy a 1 1/8th stem.

for a 1.5″ steerer you buy a 1.5″ stem

For a tapered steerer you still buy a 1 1/8th stem as it obviously tapers from 1.5″ at the bottom to 1 1/8th at the top.

You then have to choose a model with the correct size handlebar clamp or use a shim.