Hazard perception test [RANT]

How can a test designer be happy with that? It has to work in application, not just in statistical theory

What? The statistics was with regard to its practical application. Not some theoretical exercise. The data for the statistical validation was from practical application.

On any google search of forum search you find the same accusations levelled at the mechanics of the test.

of course you do, people like to complain in public. The thousands upon thousands who are quite happy with the test don’t feel the need to go to a forum and shout about it.

In fact the more I think on it, the happier I am in my opinion that it is a monkey spunk of a test regardless of the sincerity of the methodology the designers insist they used.

The methodology was not designed for sincerity, it was designed according to assessment design principles. You may be happy in your opinion that the test was no good. That’s up to you, but you have too recognise that your opinion and those of people who share similar opinions are contrary to evidence. Now in the past I have read the religious threads where people are banging on about evidence and how if some existed, then they would change their minds. I can’t present you with evidence, but I have seen it. You can choose not to believe, that too is your prerogative, and no doubt more comfortable for you. I’m not really here to argue the point. Just to let you know that this test was not some ad hoc construction. You failed it, your ego defence mechanism will only allow a limited number of reasons for this. I can’t really do much about that.

No, why would it be?

Why wouldnt it be? Is it proprietary? Is there a reason why we cant see how the test assessed for meeting it’s objectives?

So if you have nothing to fear in the methodology why not publish? Or at least publish the timing criteria for the hazard perception. Is it timed from the first discernible visual cue? Sometime thereafter? Does the timing window begin when the hazard is at some point in the field of view? ANy reason why we cant know these things because the test intro video is a bunch of bollocks on giving any guidance on when youre supposed to start thrashing the mouse button isnt it.

EDIT: OK, i’ll accept your argument. But why is it not possible to issue a “marking guide” that accurately reflects how the computer will be testing your perception awareness. No one would publish an A’level exam without a marking guide.

Stoner earlier

TandemJeremy – Member
Love it.

93% of would be bikers pass the test – perhpas because they prepared for it?

Stoner does not – therefore the test is at fault. the fact that other people whinge about it is proof

Maybe stoner, not unreasonably, thought the test would have some relevance to the real world, i.e. spot a hazard, respond right away. Not wait for the computers window of acceptance.

Why wouldnt it be? Is it proprietary? Is there a reason why we cant see how the test assessed for meeting it’s objectives?

Nope

So if you have nothing to fear in the methodology why not publish?

Because there is nothing new about it, there is no contribution to knowledge. A well established test validation process was used. Would expect item writers for GCSE to publish the findings of the pre-test trials?

Or at least publish the timing criteria for the hazard perception. Is it timed from the first discernible visual cue? Sometime thereafter? Does the timing window begin when the hazard is at some point in the field of view?

You mean tell you the answers??? If you don’t know when a hazard is a hazard, then you are the person we want to identify

ANy reason why we cant know these things because the test intro video is a bunch of bollocks on giving any guidance on when youre supposed to start thrashing the mouse button isnt it.

Have you tried the replay function? This might help you to differentiate between specific cases of hazard and non-hazard

You mean tell you the answers??? If you don’t know when a hazard is a hazard, then you are the person we want to identify

No I mean show on a demo where the test is calculating the first potential perception of the hazard. Because it’s clear that the most common reason people are missing the scoring click is because they have identified a potential hazard before the computer has opened the window for that hazard perception. I can understand it’s to minimise capturing people blindly clicking, but it’s excluding those with faster hazard perception which is perverse.

I have to retake the test with the explicit intention of delaying clicking when I identify a hazard – that’s nuts.

the fact that other people whinge about it is proof

PMSL at that one – absolute proof is obtained by the number of people whinging? Absolutely awesome mega bullshittery going on there.

http://www.theory-test.co.uk/asp/hazard_perception_info.asp#hazard_score

No I mean show on a demo where the test is calculating the first potential perception of the hazard.

That is on the review function on the demo test.

Because it’s clear that the most common reason people are missing the scoring click is because they have identified a potential hazard before the computer has opened the window for that hazard perception.

Because it is not yet a hazard. There are hundreds of potential hazards in the video, only some of them are actual hazards.

I can understand it’s to minimise capturing people blindly clicking, but it’s excluding those with faster hazard perception which is perverse.

No, it’s excluding those with an over-sensitive hazard perception

I have to retake the test with the explicit intention of delaying clicking when I identify a hazard – that’s nuts.

no, you have to retake the test and learn to differentiate between seeing something and identifying it as a hazard.

If you don’t know when a hazard is a hazard, then you are the person we want to identify

No, it’s excluding those with an over-sensitive hazard perception

Looking at the example clips mentioned earlier: I identified several hazards that were not apparently the hazard, even though I would have taken avoiding action or at least eased off the throttle and covered the brake.

So does the test also seek to identify me, perhaps as over-cautious/sensitive?

If it does then it failed, because I passed it first time 😀

Bigbutslimmerbloke – I agree – but that seems to be stoners position 🙂

In fact the more I think on it, the happier I am in my opinion that it is a monkey spunk of a test regardless of the sincerity of the methodology the designers insist they used. Not one person in this thread as far as I can see has concluded that the test itself is a good test, the principle of the test is welcomed, yes, but not the test itself – even those who have passed it first time, and even TJ (which is rightly so as he hasnt sat it).

On any google search of forum search you find the same accusations levelled at the mechanics of the test.

[stoner logic]the test must be shit because he – a driver with decades of experience failed and the proof the test is shit is loads of folk whinge about it.[/stoner logic]

Stoner logic. Is this 2012’s answer to TJ law?

Cards on the table;
Advanced driver.
Blue light ticket for rapid response cars.
Keen motorcyclist.
Practiced at recognising and differentiating hazards at high speed under pressure.
Unblemished accident record over many years of response driving (touching a big ass chunk of wood as I type)

My opinion; the test is too simplistic, and a bit pants. It has laudable intentions, but can’t replace decent instruction and practice. I had to do it for my motorcycle test and although I passed, I was disqualified on a particularly busy segment through a town centre for ‘cheating’, which I resent. I wasn’t cheating, I was following the advise to click when you spot the hazard, click when it develops, and click when you would take evasive action. There were a lot of potential hazards.

no, you have to retake the test and learn to differentiate between seeing something and identifying it as a hazard.

Firstly you have to decide which of the potential hazards is the hazard that the test is testing you on.

So does the test also seek to identify me, perhaps as over-cautious/sensitive?

If it does then it failed, because I passed it first time

It’s built on a probabilistic model. It is difficult to speak of individual cases, but it would seem to indicate that you at least identified the real hazards when they were there. Where stoner did not, unless you were under the threshold of click happiness and he was not.

No assessment system is perfect, but the statistical approach is used to say that the balance of probability is that Stoner is not of the required standard and you were. What is not available is how much you passed or failed by.

the more I think on it, the happier I am in my opinion that it is a monkey spunk of a test regardless of the sincerity of the methodology the designers insist they used.

So what you’re saying is, you think they intentionally designed it to be simian semen?

Because it is not yet a hazard. There are hundreds of potential hazards in the video, only some of them are actual hazards.

What practical benefit do we get from testing candidates’ ability to recognise actual hazards after they’ve become, er, hazardous? It’s too late by then. Surely the ability to spot potential hazards is precisely what you should be testing for? Are we really teaching behaviour which says “ignore potential hazards, just react to things”? Really?

Wow.

Cards on the table;
Advanced driver.
Blue light ticket for rapid response cars.
Keen motorcyclist.
Practiced at recognising and differentiating hazards at high speed under pressure.
Unblemished accident record over many years of response driving (touching a big ass chunk of wood as I type)

I see your problem. You’re too complacent.

STW can be edumacational, who knew.

Why is a computer, which can only work in 1s and 0s, assessing something that is by its very nature, not so black and white?

Computer says you were 0.001 seconds too fast in spotting that child.

Computer says you clicked too rhythmically and are therefore cheating.

If hazard perception is so important (which I doubt anyone will deny), then why just not assess it as part of the practical test where it is REAL world, with a real and objective assessor who can use discretion? Which it surely is anyway? In which case why on earth have we bothered to include it as a wee interactive movie screening in the theory test?

Are we really teaching behaviour which says “ignore potential hazards, just react to things”? Really?

Wow.

Which is pretty much the allegation here
http://www.hsmassociates.eu/hazardperception.pdf

• Does not realistically measure the candidates skill in scanning and
hazard perception
• Encourages reactive behaviour which is in direct contrast to the aims of
the exercise to encourage better scanning and anticipation, and;
• More important and worrying than all, scores awarded in this test do
not, in any way, help separate ‘safe drivers’ from those who could turn
out to be unsafe drivers.

I see your problem. You’re too complacent.
STW can be edumacational, who knew.

LOL. Can you test for that? I mean, statistically?

Are we really teaching behaviour which says “ignore potential hazards,

Well, we are saying not everything is hazardous, yet everything is a potential hazard. After all, there are lots of things in the videos, every one of them could become hazardous, we don’t want constant clicking.

One doesnt get tested for that, TJ passes judgement.

Stoner – what prep did you do for this? Did you do any with professional training bodies?

Well, we are saying not everything is hazardous, yet everything is a potential hazard. After all, there are lots of things in the videos, every one of them could become hazardous, we don’t want constant clicking.

Ah. So the candidate’s clicking technique when interacting with the test is of higher importance to the test’s creators than the candidate’s hazard perception skills?

It’s a good job you’re here to clear these things up. Up until this point I was thinking Stoner was simply a buffoon who went into a test without proper preparation; now it transpires that he’s probably right and the test simply isn’t very good.

Stoner – what prep did you do for this? Did you do any with professional training bodies?

You’ve got to love the technique… 😆

Well, we are saying not everything is hazardous, yet everything is a potential hazard. After all, there are lots of things in the videos, every one of them could become hazardous, we don’t want constant clicking.

This is what I don’t get from the sample page. Do I win or lose points for identifying all the hazards and extra points for identifying the hazard that I’m being tested for as it develops? Or will I be disqualified for rhythmically identifying all the hazards?

Ah. So the candidate’s clicking technique when interacting with the test is of higher importance to the test’s creators than the candidate’s hazard perception skills?

The test needs some way of differentiating between constant undifferentiated clicking and clicking associated with identification of a hazard. i hadn’t said it was of higher importance, only that it is a source of information. You could carry out a similar assessment but instead of using a computer simulation, you sat in a car with the candidate. You need to make some judgement on whether or not they were able to identify hazards, so you ask them to shout each time they saw one. If they just shouted all the time, you would have to surmise that they didn’t really know a hazard when they saw one.

So when i say we don’t want constant clicking, what I really mean is that we don’t want to pass people who just click constantly.

Less satirically,

there are lots of things in the videos, every one of them could become hazardous

When the test was being created, did no-one think this might perhaps slightly relevant? Or indeed, y’know, the entire crux of observation and situational awareness? There are a lot of potential hazards on the road which is the reason we’re testing people and the test designers figured, gosh, that’s hard to test for on a button click, lets penalise anyone who spots them instead?

I think it’s my lack of natural rhythm that must have saved me.

Up until this point I was thinking Stoner was simply a buffoon who went into a test without proper preparation; now it transpires that he’s probably right and the test simply isn’t very good.

A little from column A, a little from column B.

Do I win or lose points for identifying all the hazards and extra points for identifying the hazard that I’m being tested for as it develops

What other hazards did you identify? on which test?

There are a lot of potential hazards on the road which is the reason we’re testing people and the test designers figured, gosh, that’s hard to test for on a button click, lets penalise anyone who spots them instead?

The art is in identifying when a potential hazard becomes a real one

So when i say we don’t want constant clicking, what I really mean is that we don’t want to pass people who just click constantly.

but the mechanism you’re using to eliminate constant clicking is also eliminating those that identify hazards early – not just “ped on pavement” but “ped turning slightly as they walk past car.” That’s enough to make someone who is observant click as a developing hazard, but the test doesnt open the “developing hazard window” until later.

On any number of other threads, there’s talk of gaming the test, from non-rhythmic non-stop clicking to the one scuzz suggests: wait an extra second or two for the computer to wake up then fire off a handful of random timed clicks. Isnt it a sign of a failure of the test when you have to modify what is ostensibly a safe behaviour into a less safe one just to pass?

What other hazards did you identify? on which test?

The two I’ve looked at have a cyclist coming out of the side street and the VW Beetle picking up the passenger then pulling out.
On each video there are pedestrians who could potentially be hazards if they decided to run into the road. The cyclist video lost sight of a car at the mini roundabout which could be a potential hazard if it decides to turn right, another video had a car coming out of a driveway. These, and any car coming in the opposite direction, are potential hazards.
I would find a test that allows you to explain your actions much more beneficial.

The art is in identifying when a potential hazard becomes a real one

So it’s not like real life then.

The test needs some way of differentiating between constant undifferentiated clicking and clicking associated with identification of a hazard.

Sure, but in my position as a mere layman observer it’d seem to me that redefining “hazard” to conveniently fit the test you’ve created isn’t really the best solution.

I don’t mean to pick on you incidentally, Sam, and I genuinely appreciate you coming along to help explain. But it does seem to me (based on discussion here and the linked practice tests) that the test is flawed. And it sounds from what you’re suggesting that the reason it’s flawed centres around the fact that reliable computer-based testing is a little bit tricky. Which makes me want to type furiously on the Internet.

but the mechanism you’re using to eliminate constant clicking is also eliminating those that identify hazards early – not just “ped on pavement” but “ped turning slightly as they walk past car.” That’s enough to make someone who is observant click as a developing hazard, but the test doesnt open the “developing hazard window” until later.

and yet it is very good at predicting safe drivers, differentiates well between expert and novice drivers, and correlates well with other driving skills tests.

It might be that the test agency don’t think that a pedestrian turning slightly is particularly hazardous

The art is in identifying when a potential hazard becomes a real one

How does this help improve my driving? Can I safely ignore undeveloped potential hazards until they’re actually hazardous? That’s what the test rewards, apparently?

On each video there are pedestrians who could potentially be hazards if they decided to run into the road

sure, but did you click for the apparently parked cars which might pull out in front of you? or the cars on the other side of the road who might decide to drive into you?

How does this help improve my driving? Can I safely ignore undeveloped potential hazards until they’re actually hazardous? That’s what the test rewards, apparently?

Yes, it is a hazard perception test, not a potential hazard perception test

It might be that the test agency don’t think that a pedestrian turning slightly is particularly hazardous

How can they know? It will, after all, not have developed, unless they’ve seen the video a couple of times… 😕

why just not assess it as part of the practical test where it is REAL world, with a real and objective assessor who can use discretion?

Hazards may not occur in a real life test? Some people sit the test in central London, others on Skye. Having a standardised video makes sense in that respect.

What other hazards did you identify? on which test?

On the sample Clip 1, linked to earlier, in the first 15 seconds or so you are driving down what appears to be a narrow street with cars parked both sides – there is a van with the passenger door open, an oncoming car, and a recently parked car.

I’d probably adjust my speed or at least cover a pedal for the first two (I’d keep an eye on the third) so doesn’t that qualify them as hazards – i.e. an obstacle requiring some action from the driver?

It might be that the test agency don’t think that a pedestrian turning slightly is particularly hazardous

On the review of the sample clip, the 5 point window opens while the pedestrian is still on the pavement, turning slightly.