Air traffic control 'issue'

A dodgy flight plan is what it was. Who knew?

Matt’s colleague’s husband in NATS, apparently.

@theotherjonv - Is there an issue with being close to the edge?

Yes, The 'wake' bounces back off the wall and affects the swimmer. Another issue is some pools have (maybe had, it's been a long time since I was a competitive swimmer) a diving pool attached to the main pool and this 'sucks you in' as you swim past it.

I too googled the 'true olympic sized pool' stats and was surprised to read that the UK has 19 in total and Paris alone has 20!!!

What gets done in Spain that can’t be done in a pool in the UK?

Per the previous reply, nothing except the 50m pool. If you're swimming at national level or hope to, you need experience racing and training in 50m pools instead of the normal 25m pools. Locally, there's a 50m pool at Wycombe and one at Hillingdon. Both are fully used by their clubs so the 05:30 slot is the only one available for an hour. Previous camps have been at Mount Kelly (Hogwarts transported to Tavistock) and near Pisa.

There are more 50m pools in hot places because they don't need to build a ruddy great building over the top and heat it all.

Championships have been in Sheffield Ponds Forge and London Aquatics Centre - both excellent but fully used.

Good new though - flights re-arranged for tomorrow at 14:30, muhc more civilized.

There are more 50m pools in hot places because they don’t need to build a ruddy great building over the top and heat it all.

I'm sure that's generally true but there's not a huge difference between Paris and London weather, and yet Paris seems to have more than the whole of the UK...

It's gonna be a problem with the NATS system accepting any free text/code and then throwing a fit when the text is wrong/unexpected, probably a stray comma or something..

I'd put money on it.

If it was badly formatted flight plan instructions from a French airline, as are the rumours, then the system should have simply not acceped the instruction, rather than accepted it and crashed the whole UK system.

I wouldn't want to be part of the post-mortem/lessons learned meeting for this one 🙂

See fuzz testing - chucking garbage in in a vaguely systematic way to see what happens. Best done if the people crafting the garbage already have a fair idea of what might cause the system problems.

It may be that everyone did all the right things but shit still happens sometimes.

It may be that everyone did all the right things but shit still happens sometimes.

I don't buy that for a second.

Especially when it comes to critical systems.

As it happened the 'fail safe' worked.

But unfortunately the fail safe was to shut down the entire UK airspace, as opposed to simply rejecting an erroneous flight plan.

No idea what happened at the ATC end, but all too familiar with the problems it causes.

An aircraft is given a SLOT. That’s a take off time, coordinated between departure airfield, landing airfield, enroute ATC zones, Airport Area Control zones, etc. This SLOT means that all the above have capacity at the correct time.

Capacity can be reduced by weather , staff shortages, equipment failure etc, meaning capacity can be reduced at any 1 point from say 50/hr to 10/hr. The other 40 get delayed, which also knocks on delays as in peak times, everything is near max capacity. This is called a SLOT DELAY.

Aircraft utilisation is very high in peak season. After each flight, the aircraft is straight back out again on another.

Once the aircraft has a SLOT DELAY on one flight, it will be late for the next. Then it might pick up another SLOt DELAY, which cumulatively adds up over the day.

In this case, the ATC computer that issues SLOTS and coordinates flight plans went down, so nobody knew who was allowed to take off at which time.

The backup system is manual , which is slow and manpower intensive. This reduced capacity at all uk airfields/airspace.

The only way pilots knew when they were allocated a slot was to call their Ops centre. Imagine one person suddenly getting hundreds of calls to communicate. Phone gridlock and chaos.

These delays built and built. Pilots/cabin crew have maximum duty hours by law - longer than lorry drivers, but still legal limits.

Even when flights went, pilots and aircraft then got stuck in the wrong place. This takes days to rebuild at any time, let alone in peak summer.

Every flight this time of year is full. Capacity to rebook passengers on next flights is very limited.

I hope this makes sense.

lm currently in Barcelona. The cabin crew I am flying back with tomorrow are 4 hours late in tonight,  require their legal res5 period before flight and we will be 3 hours late tomorrow before we even start. Then I am due to fly to Glasgow afterwards. There was an hour flex, but that has gone and I will be late.

Add this across hundreds/thousands of flights and you can only imagine.

Nobody would believe a critical system could be this susceptible to falling over with so little provocation - apart from those of us that have worked with critical systems in other industries... 🙄

Someone tried to plan a flight at 1300 from CDG to LHR on 'truncate table tbl_flights' airlines

Nobody would believe a critical system could be this susceptible to falling over with so little provocation – apart from those of us that have worked with critical systems in other industries…

It's all too common, tight budgets ingoring simple things... been there, got the t-shirt, and quit to preserve my sanity.

My money is on a dodgy instruction into the NATS system, and also the NATS system crashing as a result of an instruction it should have ignored by default....

So we have 2 fails here.

£100 in Greggs vouchers.

Any takers?

Someone tried to plan a flight at 1300 from CDG to LHR on ‘truncate table tbl_flights’ airlines

So a data parsing or data format error?? In which case, surely the source system should have handled it by rejecting the input??

Makes a change from the usual hardware failure, server overload or un-tested s/w upgrade....

Does Null in Bosnia have an airport.....that's  just asking for trouble...

<p style="text-align: left;">I blame the French for filling a flight plan with one of their accented letters, nobody expected our post Brexit IT system would ever have to handle them.</p>

hahah! maybe there was a special character like '€'

Computer says no, only '£'.

It was indeed a dodgy flight plan on the French side of things that crashed the system.

I blame the French for filling a flight plan with one of their accented letters, nobody expected our post Brexit IT system would ever have to handle them

Yes, but we took back control of our skies.

After being cancelled at AMS on Monday evening my wife rebooked for this morning and has been on the tarmac for an hour with an unknown amount of time to go. Excuses so far have been bad weather and staff shortages at LGW

Pilot has said it’s chaos at Gatwick and nobody appears to be gripping the situation!

So a data parsing or data format error?? In which case, surely the source system should have handled it by rejecting the input??

I can understand how this could have happened. I once wrote some code for a major telco that was involved in connecting new customers, at a time when there were thousands a day.

I had to convert from one date format to another, and if there was something very wrong with the input format (a month that wasn't 1-12) the transaction failed. Unfortunately a stray " ! " which indicated a comment meant that one instruction wasn't run and all transactions failed when you got to December.

It was fixed pretty quickly on the 1st Dec but meant there were no connections for a few hours.

I moved on from being a coder quite quickly...

Pilot has said it’s chaos at Gatwick and nobody appears to be gripping the situation!

It was like that a month ago when I was at Gatwick anyway - apparently it is pretty much default....

Imagine having an important piece of national infrastructure that someone could take down remotely, stranding thousands of people, and costing millions of quid, just by someone in a different country sending a single file with an error in it.

Surprised the Russians or Chinese didn't get there first. But I'm sure there are lots of other vulnerabilities they can exploit whenever they feel like it.

If it was badly formatted flight plan instructions from a French airline, as are the rumours, then the system should have simply not acceped the instruction, rather than accepted it and crashed the whole UK system.

Sounds like one of those text message strings that could be sent to some phones to crash/brick them remotely or in other words "black hat hacking" through a French airline with poor security by a state actor?

That makes things interesting from an IT security perspective if our ATC system can be crashed by a specific text string what else is it vulnerable to? This time we're sat on the ground waiting to leave, next time?

Hard to believe that this is the first time a flight plan has been badly formatted isn't it? Does lead to me thinking there was a bit more behind it than that, i.e. deliberate.

Is it the first time a flight plan has been badly formatted? Or just the first time one, with just the right issue, has made it past the checks that are there to protect the system? Or should we be donning our tinfoil hats?

At this point all the official pronouncements are noticeable for what they aren't saying. (Things like hacking and state actors have not been explicitly ruled out). This maybe because they aren't involved (hurrah) or it's not certain what happened (not so hurrah).

Or should we be donning our tinfoil hats?

If you already have one then up to you. I would not be surprised if it were deliberate and it may be best not to make that common knowledge at this point for a few reasons...

We seem to have been very lucky.  Left the hotel as planned on Tuesday am for the flight back from Ibiza to LBA, got to the airport and Jet2 staff were greeting customers.  From the moment leaving the hotel I feared that Jet2 were going to send the text to tell us not to go to the airport.....didn't happen.  Got to the airport in good time.  The family in front of us said they were going to Manchester, Jet2 says, "Ah, it's cancelled, you are leaving tomorrow"...We got directed straight to the check-in desk, and it's probably been one of the swiftest transits through a terminal I've ever experienced.  Indeed, so quick that the plane was boarded to early and we had to wait for our slot as they couldn't leave early.

Feel sorry for those trapped in this, more so with young 'uns.

Wife home after 2 hour delay taking off. Pilot really put her foot down on the way back - took about 40 mins runway to runway!

As for the cause, I’d be amazed if it wasn’t a threat actor that was behind this. I don’t buy the ‘comma in the wrong place on a flight plan’ explanation - especially as no further info has been forthcoming.

Pilot really put her foot down on the way back – took about 40 mins runway to runway!

Ah the old "making up time en route" line. The fact is that modern jet engines have a very narrow window of efficient operation so the pilot can't put on the afterburners to make up time. What's usually happened is that the timetabled flight duration includes an allowance for delays and sometimes those don't happen.

Though when I worked on ATC research I was heading back from Brussels with a bunch of LATCC high ups. It was a long day and we were due back at peak time but they put in a call to their colleagues and pulled a few favours - the crew were visibly surprised when the pilot called 10 mins to landing.

winston

As for the cause, I’d be amazed if it wasn’t a threat actor that was behind this. I don’t buy the ‘comma in the wrong place on a flight plan’ explanation – especially as no further info has been forthcoming.

Not sure why you'd jump to that conclusion. It's perfectly plausible that accepting a malformed or otherwise incorrect flight plan has occurred due to a software bug. Testing software on any reasonably large codebase is (extremely) difficult.

Microsoft can't achieve bug free software, Google can't achieve bug free software, Apple can't achieve bug free software, Boeing can't achieve bug free software, NASA can't achieve bug free software, etc etc. It's hard.

I blame the French for filling a flight plan with one of their accented letters, nobody expected our post Brexit IT system would ever have to handle them

See, you’ve written tomorrow's Daily Wail headline.

As for the cause, I’d be amazed if it wasn’t a threat actor that was behind this

I dont know. If it had been an attack I would have expected it to stay down for considerably longer than just part of monday (unless their dr is really, really good) both to figure out how to fix and to make sure its secure.

If it was badly formatted flight plan instructions from a French airline,

Probably created using French software that had bouncy flash animations, hidden menus, dead links etc like all their freakin websites.

but you can’t blame people for flying when its a 3rd the cost. Got to love big oil
I was looking at getting the train to Toulouse in October. £278 return compared to £38 return on Ryanair.

Aren’t the TgV electric? Just asking…

TGVs are indeed electric and not cheap. There's a mass of infrastructure to maintain whereas planes fly in th eair burning tax free kerosene. That's wrong IMO. Fossil fuels should be taxed in proportion to the damage they cause so kerosene would become a luxury only a few can afford.

Probably created using French software that had bouncy flash animations, hidden menus, dead links etc like all their freakin websites.

Do you have an example? The majority of websites I use are French and of the tabs curently open there's none of that - bank, leboncoin, e-mail, social security, santé, TV, Géoportail, various building suppliers, France Bleu... .

But yeah, if there's problem, blame the French, it's been official British policy for centuries. 😉

Fossil fuels should be taxed in proportion to the damage they cause so kerosene would become a luxury only a few can afford.

Good luck with that and enjoy the skyrocketed prices of good that are shipped by air and sea!

TGVs are indeed electric and not cheap. There’s a mass of infrastructure to maintain whereas planes fly in th eair burning tax free kerosene. That’s wrong IMO. Fossil fuels should be taxed in proportion to the damage they cause so kerosene would become a luxury only a few can afford.

Completely agree.

I was reflecting on the way in this morning - people are getting very uptight around ULEZ and similar. Yet they have seen *nothing* yet when it comes to impact of climate change or paying the full cost of fossil fuels / high CO2...

TGVs are indeed electric and not cheap.

Are the French better with family / group tickets than the UK? The single biggest thing that puts me off train travel in the UK is that if more than one person needs to take a train journey and they aren’t named together on a railcard, it immediately doubles the cost whereas the expenses for a car journey are flat.

I’m more than happy to tolerate some inconvenience for a train journey, or a greater cost, but not really both.

The biggest driver to changes in air travel from an environmental perspective will be if businesses had to account for the carbon emissions of their employee travel. So that will either encourage more remote working / meetings, or force airlines to embrace SAF and greener policies.

Not sure why you’d jump to that conclusion. It’s perfectly plausible that accepting a malformed or otherwise incorrect flight plan has occurred due to a software bug. 

Little Bobby Droptables grew up and is now Capt Robert Droptables...?

politecameraaction

Little Bobby Droptables grew up and is now Capt Robert Droptables…?

Monseuir

When it was reported that there was bad data in the flight plan you'd think it was corrupt or something to cause the system to crash, but no the plan had two identical waypoints (or identically named) which brought everything to a standstill.

It’s surprising because duplicate waypoint names are actually pretty common.

It’s surprising because duplicate waypoint names are actually pretty common.

Really? I was under the impression that waypoint names are chosen so as to be unique & also not to sound alike. Not that I had much to do with flight planning or much knowledge outside my local airport area but I would’ve thought this was a basic human factors issue.