[Closed] Self Driving Car Darwin Award

The feature didn't work

irc - I agree with most of what you say, but to compare Tesla miles to "UK driver miles" is bad analysis. You can't say "UK drivers are safer", only possibly "driving in the UK is less likely to result in death than the US". They typically drive different vehicles, their road layouts are different, their driver training and law enforcement is different and potentially even their emergency response is different Its just as bad comparing UK and US stats and attributing that to the person or system controlling the velocity of the vehicle as it is to say "1 death in 130M miles to date means the death rate is 1:130M miles".

And of course that works the other way round too - just because they prove technology in wide straight roads in sunny california doesn't mean you can assume the same safety record on narrow twisty back lanes of somerset on a misty day.

irc - I agree with most of what you say, but to compare Tesla miles to "UK driver miles" is bad analysis.

Point taken. But the gist of my argument stands. One USA fatal per 95 million miles or thereabouts for human drivers is an accurate stat. 1 fatal for 130 million miles for Tesla isn't enough data. We can't say whether the Tesla system as it stands is safer. The advantages of the driver assist features may be outweighed as drivers switch off. We don't know.

oh, and Tesla agree that this system is not fully ready yet

So how the hell is it allowed on the public roads? Has it passed a driving test?

at no point should you be using it as a automatic driving system. The failure yet again is the driver.

Or alternatively it's a Tesla failure for not having some system to ensure the driver is paying attention. A modern version of the dead man handle on trains. What it should be? Who knows. Iris scanner? Blink sensor? But an company that semi automates then blames the driver for losing concentration is just shifting blame.

The other issue is driver de-skilling. First generation drivers have years or decades of experience. Drivers in an auto car world won't have that. So full automation - not an issue. Semi auto - the drivers will be worse when they need to take over.

Manual control is a highly skilled activity, and skills need to be practised continuously in order to maintain them. Yet an automatic control system that fails only rarely denies operators the opportunity for practising these basic control skills. One of the consequences of automation, therefore, is that operators become de-skilled in precisely those activities that justify their marginalised existence. But when manual takeover is necessary something has usually gone wrong; this means that operators need to be more rather than less skilled in order to cope with these atypical conditions. Duncan (1987, p. 266) makes the same point: “The more reliable the plant, the less opportunity there will be for the operator to practise direct intervention, and the more difficult will be the demands of the remaining tasks requiring operator intervention.”

An experienced driver today is probably competent enough to monitor a self-driving car but what about a driver twenty years from today who will likely not have spent any meaningful amount of time driving a manual car?

http://www.macroresilience.com/2013/05/09/deskilling-and-the-cul-de-sac-of-near-perfect-automation/

https://www.teslamotors.com/models
From my reading nothing says you don't actually need to pay attention...

[quote> https://www.teslamotors.com/models
From my reading nothing says you don't actually need to pay attention...

Operating manual? Human nature? Which will it be?

An experienced driver today is probably competent enough to monitor a self-driving car but what about a driver twenty years from today who will likely not have spent any meaningful amount of time driving a manual car?

Note also the weasel words "experienced" and "highly skilled" - neither of which are prerequisite for driving a car.

At the opposite end of the spectrum, when Ford first introduced their adaptive cruise system in the new F150, the system would slam on the anchors when met with dark shadows under overpasses on bright days... Caused quite a few wrecks there too! We are on our 4th car that has adaptive cruise (2nd with lane-keeping assist steering) and whilst it does work well, it makes me pretty uncomfortable, and I would never pay any less attention and use it as a crutch! My mother-in-law has fallen asleep at the wheel of her Mercedes ML and the system kept her on the road an woke her up 😯

I've used office computer systems for thirty years. Based on that experience there's no way I would trust my (or anyone else's) life so directly to a computerised system.

I know we are forced indirectly to risk our lives (computerised control of hazardous processes etc) but just because "we" can do something by IT does not mean that we should.

I know we are forced indirectly to risk our lives (computerised control of hazardous processes etc) but just because "we" can do something by IT does not mean that we should.

It's a classic case of launching an immature system full stop. Tesla can hide behind the fact that 'the driver is always in control and should monitor the system at all times' is a cop out. The causes of most accidents is drivers not being alert - well if they're not alert when actually driven then they're going to be even less alert by simply overseeing an automated system. If the driver is the weak link the system has to eliminate the weak link or you're not addressing the danger. The sort of half-way house driver assist systems that Tesla has introduced are just a stepping stone to fully automated driver-less system, and therefore effectively developing the system in the real world.

Unfortunately there is no global framework for allowing new technology for cars to be proved, certified and released and introduced in a safe way like there is for the aviation industry where there are global aviation authorities that regulate and oversee everything. That hasn't been an issue because before now we've been talking about new tech such as electric windows and sat nav - i.e. technology who's failure will not result in accidents. This is something different and requires a more thorough approach, other than driving around for a few million miles and declaring "that'll do". Where is the thorough global standards that the system has to achieve other than a testing To Do list that a few engineers within Tesla come up with. There is the testing to ensure the system will 'fail in a safe way' when certain critical components within the system fail?

Jesus RIP, it literally be a combination of the driver was part watching, keeping an eye in the road and using the driving feature. This combined with a tragic traffic move = a sad fatality. I just love news media painting a bloke as 100% irresponsible. You can use a feature for a while and start to trust if you know.

On a road with impulsive and sudden lane changes, chain reaction moves with no irrational thought a computer can not beat a driver IMO.

Last week I witnessed a truck driver monster and almost crush a car driver for a small slight- it was ridiculous. Would a computer spot and respond to that or throw a spanner/fault move?

I just wish the media would wait for 100% facts before slandering a dead man but then they wouldn't say '6 months ago a man was killed and here is our follow up story with all the facts now'.

As wobbliscott says.
These systems should undergo the same certification that a new passenger plane has too, which is a bit more than fly it for a few 1000 hours and saying that'll do.
Also, people seem to be too trusting of technology. The GPS is always right. There'll always be a mobile signal. The self drive works and I can read a book on the way to work.

To those saying that they wouldn't trust their life to an automated system on the basis on using a Windows pc, or that new tech up to now isn't safety critical (sat navs etc), then how much control do you think you have driving a modern car with an automatic gearbox? The engine is fly by wire and the shift lever has no mechanical link to the transmission for selection of neutral.

There are standards that if aren't followed the manufacturers will find themselves liable. Tesla are in a very grey area by referring to this functionality as beta, In my opinion it should not be present on a production car if it is genuine beta software.

I agree, the grey area seems to be getting bigger, and no-one seems to be regulating it. Engines being drive by wire is, I think, just about ok, but combine that with self parking systems (which must therefore have some degree of steer by wire and brake by wire) and I'm having to place a lot of trust in the hands of the hardware and software engineers. I'm not sure I'm ready for that yet without someone checking what they're doing. (Or I'm just getting old.)

how much control do you think you have driving a modern car with an automatic gearbox? The engine is fly by wire and the shift lever has no mechanical link to the transmission for selection of neutral.

Even with adaptive cruise control and lane assist, that responsibility lies with the driver. Apparently it is the same for the Tesla, but, people are lazy...

In the end removing the driver is the safest thing, as much assistance to correct the mistakes of the driver are good. But a driver who ignores the road is an idiot and responsible for what happens

The inputs to the engine and gearbox are purely electrical though. The software decides what to do with those inputs, there is no mechanical override other than obviously the brakes - but in hybrid and electrical cars they are becoming increasingly computer controlled due to energy regen.

The point is that we already have a high degree of automation, which if not properly implemented and standards followed, could be dangerous. Teslas autopilot system implies that the driver doesn't need to take any notice (even if that isn't the case, that's how it will be interpreted), and this is a step too far.

My issue is it either has to be fully automated or not automated.

Once you take away the steering and the throttle and the brake - the drivers going to get t bored become inattentive and not pay attention.

Dead man switch like trains ?

I do think it is coming and it's got potential to be a great thing but the ethical dilemma in such cases - so I'm in my automated car I own it's driving automatically and it kills someone.

Who's responsibility ? I reckon I knkw the answer but there will be a dispute I'm sure.

Whenever someone quotes "Darwin Awards" in relation to someone's death, it demonstrates an appalling lack of empathy and taste.

Perhaps we can nominate the OP to explain the concept of 'Darwinism' the victims' children.

The inputs to the engine and gearbox are purely electrical though. The software decides what to do with those inputs, there is no mechanical override other than obviously the brakes

It's about who is [i]in charge, [/i]and who is [u]responsible.[/u]

V8ninety - I see your point about responding to drivers inputs - if you press the accel pedal halfway, then you might reasonably expect the software to implement this for you. This should be the case but don't be under an illusion that you have direct control, a software bug could prevent this from happening. Fortunately there are standards and procedures followed to mitigate this risk.

In this example though the driver makes the concious decision how far to press the accelerator pedal. Same with steer by wire, and brake by wires systems- they should respond to what the driver is asking for. The problem that I see with autopilot is the computer becomes the driver and makes the combined decision on how far the press the accel pedal, how much to steer, how much to brake. A huge amount of subconscious decisions and observations are made during driving which I don't believe a computer is able to replicate yet, or maybe ever.

Automotive software isn't just hacked together in someone's basement like a lot of everyday software is, it is a highly regulated environment that goes to great lengths to demonstrate correctness at every stage of the process. The more complex or safety critical a function, the more nauseating the certification process, so I have no doubt that a huge amount of effort has been put into making the Tesla system as safe as they can make it, and that will continue into the future as issues like this surface.

While only "beta" this will still be far better designed and tested than 99% of software out there will be. The various agencies such as the US NHTSA wouldn't let them sell it if it wasn't.

Too late - I agree.

Perhaps we can nominate the OP to explain the concept of 'Darwinism' the victims' children.

The software required to drive a gearbox is many levels of orders of magnitude less complicated than that required for an autopilot system. You can predict quite simply the number of different possibilities and situations that an auto gearbox has to deal with - maybe 20 or 30 different scenario's. But the number of potential possibilities that an autopilot system has to be able to deal with will add up to the millions - almost infinite - so many that it is impossible to hard code into software so you're looking at incorporating elements of artificial intelligence. The two are not even comparable.

I'm not saying that they are comparable, but what I am saying is that we already have a large amount of automation in cars at this point in time. People on this thread are suggesting that based on sat navs and windows computers that they wouldn't trust automated systems, but the truth is that they already do. A software bug in engine software, gearbox software, etc could put people in danger, but we do have standards that have to be adhered to, so that this risk is mitigated.

You are right, the decisions that a gearbox has to make are many many fewer than autopilot, but incorrect gearbox software could still result in tragic consequences.

if you press the accel pedal halfway, then you might reasonably expect the software to implement this for you...but don't be under an illusion that you have direct control, a software bug could prevent this from happening

Since machines have become increasingly complex, we have had less and less 'direct' control. The paradigm shift that is going on now though is that we are actually handing [i]fundemental control[/i] to the machines. As a driver (responsible for my own errors) I'm not sure I'm ready for that yet.

incorrect gearbox software could still result in tragic consequences.

The Tesla software will be more complicated, but all of it will still have been through the same extremely rigorous certification process; I couldn't even begin to imagine how time consuming and expensive that must have been, and I'm a software engineer who has worked on much, much simpler safety critical systems in the past. It's a phenomenal amount of work.

I bet you're still more likely to get killed by the failure of some really boring bit of hardware like a bonnet catch or something than any software system.

phiiiil which would make it even more frustrating for the engineers having their work shot down by a journalist who half baked stories for a living

The difficulty in the step up to fully autonomous control is actually why I prefer Tesla's approach than Google's; we are generally better at improving what we have in tiny steps than coming up with a finished product in one go.

Both are producing extremely valuable results so I'm still glad people are approaching this from many angles, but the incremental approach makes it less of a shock.

phiiiil which would make it even more frustrating for the engineers having their work shot down by a journalist who half baked stories for a living

When some VW bigwig a while ago blamed their test cheating on "a couple of rogue software engineers" he was talking absolute bollocks; there is no way in hell you could ever sneak anything in to that kind of software without armies of people knowing what you're doing.

Since machines have become increasingly complex, we have had less and less 'direct' control. The paradigm shift that is going on now though is that we are actually handing fundemental control to the machines. As a driver (responsible for my own errors) I'm not sure I'm ready for that yet.

I've often said that if you invented the car today you'd never be allowed to introduce it onto the roads. The big challenge in autonomous vehicles isn't driving the vehicle, its guessing what the non-autonomous vehicles (and perhaps to a lesser extent animals, peds and bikes) might do. Interestingly I believe my grand children will never actually drive a car (if I have any they won't be driving for about 25 years at least), and to make that big leap happen it might be necessary for us to do something radical like banning normal cars from some sorts of road, or insisting they all transmit certain data to help make them predictable to their near neighbours. My grandkids will look back at the improved safety statistics and wonder how it took us so long to phase in the technology.

The Tesla software will be more complicated, but all of it will still have been through the same extremely rigorous certification process;

Yes but who's certifications process?
Teslas? or someone independent?
Planes have to be certified to external standards (CAA, FAA etc). Is there an equivalent for self driving cars?

Technically, if anyone is explaining Darwinism to the deceased children, then it doesn't actually apply... Jus' sayin

That was my point

Yes but who's certifications process?

From a safety point of view it doesn't matter all that much what a system does, but you have to demonstrate that you've spent ages and ages and ages doing things properly, testing it thoroughly and covered every eventuality you can think of. This will be a very long list with automatic driving systems.

Inevitably there will be things that get missed, like this, but that doesn't mean the process doesn't work; things still crop up occasionally with aircraft systems and they are more mature and even more risk averse than cutting edge automotive systems like this.

Also worth noting that this accident probably wouldn't have been fatal had it occurred in Europe where trucks have side underride bars, and possibly wouldn't have happened at all

Not true - I suggest you read the side under-run regulations before quoting that one.

The side bars only have to resist a very small force - basically to stop pedestrians or cyclists going under the rear wheels. Some trucks do have beefy side rails, but they aren't legally required. Most trucks just have flimsy rails or a GRP moulding attached to skinny galvanised steel brackets. Rear under-runs on the other hand are tested to a very large force and have a certain contact area, height etc.

@phii no it was a demo setup with a model pedestrian and the car was suppsoed to stop but it didn't

As I said Darwin awards have been doing the rounds for what 10 or 20 years ?

The guy wasn't paying attention, he was replying ona system which Tesla tell you not to reply on but its quite clear to me people will reply on. I've seen some stats about the number of interventions there have been with the google cars, many incidents where without that there would have probably been a crash.

Product safety regulation of the like we see in the Aviation industry are completely different to standards bodies which are largely voluntary. For a start they are compulsory and fully regulate all the elements of the industry from aircraft and aircraft systems design, to the design of airports, the air traffic control systems and right down to how operations on the ground at airports behave and how airlines have to manage their paperwork, maintenance and even stock and inventory control of spare parts. This is nothing like what bodies like ISO, MISRA etc do. They don't even scratch the surface in comparison.
If we're going to move towards fully automated self-drive cars then the whole transportation network has to be brought under a common body that see's over the design of the road network, the vehicles themselves, absolutely everything.

That might lead to proper segregated cycle paths...... only by virtue of if we are allowed to mix with automated cars the cars will be crawling along confused at cyclists 🙂

Someone pointed out to me today the utterly fantastic yet also horifically tragic outcome of self driving cars will be that in the future, sooner or later, someone is going to die of natural causes behind the wheel.

So, someone will be expecting their wife/husband to come home from work, or their dinner guests to arrive, and a car is going to pull up on the driveway with their loved ones dead body inside for them to discover 😯

You could just enter the postcode for the funeral directors and send em on their way..
Drive in mausoleums could be the next big earner too, laid to rest in a Tesla..