My folks took me to a ski resort for the first time when I was 4ish. Gran looked after me whilst the folks went skiing. A couple of years later I joined in the skiing too and have been for (nearly) every one of the 30 years since and can heartily recommend it.
HOWEVER, since you did say you don’t want to do the whole ski thing, be very careful what resort you choose. I wouldn’t recommend going to the Sierra Nevada if you’re not going to ski. It can be very cold and very windy and is not the most interesting resort to just “mooch around”.
I’d recommend Chamonix. You could probably do it quite cheaply, there’s loads to do other than ski/climb and you’re beneath the highest mountain in Europe. The cable car up to the Aiguille du Midi is particularly impressive and will either exhilarate or terrify the kids. On the downside it is not what you’d call “quaint”.
Avoid the purpose built French resorts. They are great for skiing but invariably ugly. Go for a traditional resort, which the Austrians seem to be good at, but make sure it ticks the boxes for the non-ski activities e.g. swimming pool, ice rink, sleigh rides etc etc.