The problem is that blanket limits only teach drivers (and we are all drivers, it's not "them" and "us") to ignore those limits. Think, about your driving history. Every year you drive thousands of miles without incident, and you do that at a speed that feels appropriate to you. Suddenly sticking up a 20mph sign doesn't change how you view the road in front and around you. In fact, a dumb 20mph sign just sits there, shouting out a message that your very eyes can see, for the vast vast majority of the time, to be completely erroneous!
The simple fact of the matter is that the driver who crashed outside your house at "50mph" didn't crash because they were speeding, they crashed because they were a terrible driver. Our recent obsession with speed limits, (and machine justice), focusing on some completely arbitrary value has mean't that there are more and more "bad" drivers on the roads, often who drive at or below the speed limit and so are practically invisible as far as the system is concerned. Everyday you will see poor attention, inappropriate speed (often below the posted 'Limit'), inappropriate vehicle positioning, poor lane discipline, failure to indicate and use mirrors, tailgating etc etc. Since we took real policemen and women off our roads in favour of speed cameras, those supposedly minor offenses have, imo, been left unchecked. And once you get into the habit of poor, lazy, inconsiderate driving, doing 50mph down someone else's street (or often even your own!) doesn't seem so bad.
Add to this modern cars, that now completely isolate the driver from the outside world, and take little of no skill whatsoever to operate, and becoming a lazy driver is the norm. You simply don't have to think about driving any more, just do it.
So, whilst driving is considered just a means to getting from A-B between all the other more important things you needed to do today, rather than a proper task in itself, then people will continue to drive at inappropriate speeds, whatever the actual speed limit is set too...........