Subscribe now and choose from over 30 free gifts worth up to £49 - Plus get £25 to spend in our shop
After a great ride today, I am outraged that I can't see how awesome I was.
Nip on Garmin sort it out
Here's the status page: https://connect.garmin.com/status/
Literally everything they do is down, and has been all day.
According to the app it is down for maintenance but that has been since 07:30 (probably before then) so I suspect someone has properly broken something for it not to be back up by now!
Hope it sorts itself soon...I've got some scales to register...I know I'm fat but I want to know the breakdown of how fat!!!
OMG....
Hopefully sort out the sleep tracker and the issue to why the steps on my phone (Apple health) don’t match with my watch
Put your phone on your wrist next to your watch - the movement of your wrist will count on both and your steps should get a closer match.
Load it up to Srava to see how awsum you are 🙂
After a great ride today, I am outraged that I can’t see how awesome I was.
Nip on Garmin sort it out
What ride? If it ain't on strava...
C'mon garmin, I need to see my training status!!
The longer this went on, the less likely it was to be a typical failure and it is starting to look like this is indeed the case.
A Taiwanese IT website is saying that it is a ransomware attack and that they've shut production down (in Taiwan) as a result.
iThome received news from readers that Garmin (Taiwan International Avionics), a well-known GPS and wearable device manufacturer in Taiwan, was suspected of being hacked. The IT department sent a notice to various departments in Taiwan stating that internal IT servers and databases were attacked and production lines were also suspended. Two days. It is speculated that Garmin may be attacked by ransomware.
Bad day to be in IT at Garmin, I would say... :-/
Put your phone on your wrist next to your watch – the movement of your wrist will count on both and your steps should get a closer match.
Heh! My phone is actually recording more steps and is accurate, the watch has been thousands out
If it is ransomware
I hope all their different countries etc are fire walled from each other and it’s not got into their whole system. Otherwise they’ve got big problems - they might not even get full access if they pay the ransom.
Travelex got hit earlier this year and I’ve seen a number of large ish companies lose access to their entire network as a result. You either pay up and hope for the best or accept you’ve lost everything and completely rebuild your systems.
I've ended up downloading from watch onto laptop, then strava, I am indeed oarsum
I hope all their different countries etc are fire walled from each other and it’s not got into their whole system. Otherwise they’ve got big problems – they might not even get full access if they pay the ransom.
Pretty sure GC is one, big platform and not individual regional deployments. This is based on when it's gone down in the past before, it's always been global. The worry is whether whoever compromised them also downloaded a copy of the data because there a lot of *seriously* personal data held in there...
Their Facebook message mentions their call centres are down too - no email / phones etc which doesn’t bode well
Their Facebook message mentions their call centres are down too – no email / phones etc which doesn’t bode well
No, indeed. I suspect that most people working in IT would have read that and started thinking "ransomware" immediately.
Think about how long Travelex were down - they were offline for way more than a month and they were even reported to have paid $2.3m to recover their systems too...
Manual uploads to Strava it is then
If it is ransomware
I hope all their different countries etc are fire walled from each other and it’s not got into their whole system. Otherwise they’ve got big problems – they might not even get full access if they pay the ransom.
restore from backups is the answer - they obviously take backups dont they, nobody would be silly enough to think that just because its on the cloud they dont need to take backups.......
Assuming their backups haven’t also been encrypted if this is a ransomware attack....
I've just bought a set of the Garmin Smart Scales (which I had to wait for about 6 weeks to arrive.)
Been using them for a week. Really cool, uploads all of your stats to Garmin Connect.
Except now they don't. Ar$e.
Assuming their backups haven’t also been encrypted if this is a ransomware attack….
...and their SCM / software repository. If they are rebuilding from scratch, just getting the platform back up (even without customer data) might be a challenge in itself if they don't have access to their own software!
It'll be a good test of whatever business continuity planning they have in place and how they handle the coming hours and days will be a very public assessment of these processes. Travelex failed badly, maybe Garmin will do better...
Assuming their backups haven’t also been encrypted if this is a ransomware attack….
Ah - the benefits of offline backups.
I'm going to guess that they've had a ransomware attack that has encrypted a few computers/servers (probably end user laptops/desktops) , but they're not confident that the rest of their estate is well protected enough to withstand attempted attack from inside their network, so they've taken the lot offline while they work out how to protect the stuff that hasnt yet been affected.
We'll probably never know the full picture though...
Hopefully we will though as there is a lot of personal data stored with them, so it would be good for them to be open with their users.
Ah – the benefits of offline backups.
This can be very, very challenging on systems with very high ingest rates like telematics platforms (which, oddly, systems like Garmin Connect and Strava are most closely aligned with). Certainly the days of tape backups are long gone - I'd bet good money that you couldn't write out to tape libraries fast enough to keep up with the ingest of new data on GC.
Plenty of other options for "disconnected" backups, all of them offering a lower level of protection than "tapes in a safe", but hopefully they've got something in place. Not sure how cloud-native they are, so it may even be that they have frozen archives of data, but if (for example) an AWS root console has been compromised, then they are probably stuffed whatever they have done :-/
Ah bugger. My Tacx trainer is bricked and waiting to hear about a warranty replacement.
Just spent the last 4 weeks waiting for a part and on first use since something else let go.
Good job there is still real life out there with like woods and trails and roads and stuff. If that all breaks I am totally screwed.
Manual uploads to Strava it is then
Which is what I do as standard, GC is a truly awful piece of software. I bypass all of that and just put it straight onto Strava.
In what way is GC awful? I've had Garmin watches in various forms for 4 years, this is the first time I can recall not being able to load my ride onto Strava via GC.
It's normally seemless, and is on Strava by the time I've put my bike away to be fair.
GC is a truly awful piece of software.
True the Garmin site is pants, but "normally" I stop and save ride on watch by the time I've opened the car it's synced with Strava and its uploaded. Pretty seamless
Manual uploads to Strava it is then
How'd you do that then? Asking for a friend.
Noticed that the Garmin website is down too.....
I don’t mind Garmin Connect -‘it normally works fine. Although when it does an overnight update it seems to forget both my garmin devices (edge 530 / forerunner 645) and the only way to reconnect is to forget both Garmins on the phone and delete them from Connect and start from scratch. Happened 2 or 3 times already this year and it’s very annoying. Didn’t have anything like that with my Lezyne in several years.
Plug unit into computer - it appears as a drive. From Strava website click on the plus sign to the right of you avatar/photo and select "manual upload". Navigate to your unit and find the "activities" folder, select your activity and click OK.
I have to do this with my Oregon as it doesn't connect via bluetooth. I sometimes have to do it with my 520, just depends if it and my phone are talking to one another. No big deal as I need to charge the unit anyway.
When it’s back up, just turn on your device and sync it then then the ride will upload.
That’s what ill be doing with today’s ride.
I don't tend to use the software on my pc, mainly the app (Android) - which seems to work well.
Watch has WiFi so installs updates without a 3rd app/device and after a ride, it seems to skelp through the sync process quickly.
Segment creation on Garmin is mince compared to Strava but otherwise, Connect has been fine for me.
The 520 via the Apple version seems to take an age to sync...it connects to phone near instantly but still takes about 2 minutes after connection before it will sync the ride...that is annoyingly slow.
When it’s back up, just turn on your device and sync it then then the ride will upload.
That’s what ill be doing with today’s ride.
There's a good chance that may be days (or even weeks) away if the rumours circulating are true, but otherwise you're quite correct - it should sort itself out eventually.
Just plugged my 530 into my Mac mini and the Strava upload was easy and quick. Just not as quick as it just working automatically through Bluetooth to my phone.
Whats the link between tacx and garmin, tacx seem to think the problem is theirs. Are they part of the same group?
edit: Well a quick google says yes. https://tacx.com/garmin-acquires-tacx/
Maybe not ransomware then, could just be a ****ed up attempt to link the two companies IT infrastructures.
Typical. My new Garmin instinct watch has arrived this afternoon and I can't finish setting it up. Gutted.
Doubly annoying for me, we have our life insurance with Vitality, MrsMomo tracks her activity on a Garmin watch and I record my rides on an edge 1030, vitality app syncs with GC, so no points (or rewards!) for us until this is resolved (well, I can record on my Apple Watch so not quite so much of an issue, but still...)
As for uploading to strava, the USB ports on my MacBook have committed Harry kari, provide power but don’t do the data thing anymore, and my work laptop is locked down so can’t even use that.
Did my run today even happen?
I’m no longer sure.
In other news, since my run I have had a smoothie, some squash, then wine. And water. Now, more wine. I now consider myself rehydrated.
As for uploading to strava, the USB ports on my MacBook have committed Harry kari, provide power but don’t do the data thing anymore
Tried this?
https://smallbusiness.chron.com/reset-usb-ports-mac-laptops-51619.html
Or this for usb c
https://www.google.co.uk/amp/s/www.imore.com/how-troubleshoot-dead-usb-c-ports-your-macbook%3famp
Well that's my weekend ruined, I was going to go riding but without the kudos of other bike riders what's the point? I could manually upload but that's extra effort.
Ransomware attack is story on ZDNET and they might be spending a few days rebuilding systems. One of my favorite car forums was destroyed by ransomware, loads of good info lost.
I’ve always wanted sites like Garmin Connect etc to offer the option to upload rides but to only store metadata and summary information (time, date, distance, average speed etc), to give a log of rides without the actual geo data. This is absolutely one of the reasons why.
I see that some employees are supposedly attributing it to "WastedLocker" - a recent encryption attack tool. One positive of this is that the reports of the team behind this indicates that they are not interested in data theft (often the threat of publishing private data is more effective than threat of losing it), only in financial extortion. This suggests (hopefully!) that we won't all see years of our rides appearing on bittorrent any time soon, and that it's very much down to Garmin to either pay up, restore or fess up that they've lost a lost of customer data!
I'm guessing that we'll see it come back up sans-data first to allow new stuff to be uploaded, and then recovery of data history will follow later.
If they have lost everything and don’t get given it back it could be weeks / months before everything is back rather than day tbh. We’ve had an IT security company come in to present on a number of occasions to the large corporates we look after and it just sounds horrendous the damage that can be done. There was a solicitors business that basically only operated on personal phones using WhatsApp for weeks. Every company device was locked / encrypted and unusable. I don’t think from memory they paid the ransom and spent months having their system rebuilt in a secure way.
Let’s hope Garmin had a good IT budget and used it sensibly....
Agreed...given the volumes of data they'll be crunching 24/7, there is every hope that their systems were managed and supported properly.
Let’s hope Garmin had a good IT budget and used it sensibly…
Well, if my experience of their device firmware over the past 20 years is anything to go by…
Is this why I’ve had two rides this week not record/upload properly then?
🤬
As for uploading to strava, the USB ports on my MacBook have committed Harry kari, provide power but don’t do the data thing anymore, and my work laptop is locked down so can’t even use that.
Same here. Google cache from Garmin forums reckons it's the cable - most don't work some do. Somewhat sceptically I tried a few cables until eventually....bingo!
So try a few cables...
Annoying know. I wish the data was all stored locally and the online stuff merely a copy.
It’s affecting flyGarmin too - soon the aviation databases will be out of date and stop aircraft going flying.
Mostly small ones, but still - it’s not all dubious 5k PB claims!
Just seen this on another cycling forum:
Knowing Garmin, it was probably just a new version of their own software that they pushed to their servers.
There is/was a new software update due
Update of what though? My Fenix 6 updated last weekend and from what I can tell updates on that tend to be quarterly...was it Connect getting an update?
I appear to have a real interest in this and I'm not sure it is purely down to the fact I'm wanting to get these new scales set up!
was it Connect getting an update?
No. Despite Garmin's silence on the subject, there's been a steady set of leaks both from their manufacturing in Taiwan, and their HQ in Kansas City (yey for anonymous Reddit!) saying that it's a ransomware attack. Obviously still relatively little information on the scope of the attack - for all we know, they might have turned Connect off to protect it - but the fact that they lost their telephone system too tells me that their problems may well run deep and that it won't be coming back quickly - one doesn't turn off customer services when you're in the middle of the biggest service outage in the company's history!
My guess is that it'll take a few days at best, but after 24 hours they've surely got to make some form of public statement on this today!
It's getting on a bit for Garmin to remain quiet on this. Lot of personal data at risk if this is ransomware.
I appear to have a real interest in this and I’m not sure it is purely down to the fact I’m wanting to get these new scales set up!
I had the scales up and running for a week! really good while they were working. I can still get the readings with GC down, however all that I can do now is make a note of the weight reading and wait until GC comes back up and manually add the weight readings for the dates when GC was down.
Annoyingly its only the weight stats that can be manually added, not all of the other ones.
DLA Piper were the law firm I was thinking of that got hit by a ransomware attack in 2017. Took them 2 or 3 days just to get email back up and running and even then without emails from before the attack pretty much. Took them ages to get any previous documentation and systems operational.
Garmin must have quite a number of systems in the background to support what they do.....if the ransomware hackers got in and looked round for a few days identifying all the end points in the systems and backups etc this could be devastating.
Depending on how long this outage lasts, next time the scales sync they do seem to remember a few past readings - I know when I have had wifi issues previously more than just the latest weight reading uploaded.
I'm assuming as it can't connect, the scales can't be registered so won't do much...so it is a waiting game for me, but looking at all this, I've a few things that rely on the Garmin sync, so those will all be impacted as well for a bit.
Annoying know. I wish the data was all stored locally and the online stuff merely a copy.
Quite. I used to be quite particular in doing exactly this myself, but I got lazy. I think I might go back to my old ways. I've only ever used Garmin as a way to get stuff into Strava.
Security is hard, but given the quality of the software they put out, I'm not surprised to learn that Garmin aren't very good at it.
Having been burnt in the past with devices that need a connected service to work I try to get stuff that will work autonomously, if with slightly degraded functionality sometimes. At least it's pretty easy to get ride data off most Garmin devices without Connect.
A bit of info on the suspected ransomware from Malwarebytes;
WastedLocker is a new ransomware operated by a malware exploitation gang commonly known as the Evil Corp gang.
The attacks performed using WastedLocker are highly targeted at very specific organizations. It is suspected that during a first penetration attempt an assessment of active defenses is made and the next attempt will be specifically designed to circumvent the active security software and other perimeter protection.
The main Garmin site is back up for me (UK & US) and I can get into my account (anything I should be doing to protect my details?)
Is does seem to be Connect/Express specific at the moment.
PJay
Member
The main Garmin site is back up for me (UK & US) and I can get into my account (anything I should be doing to protect my details?)Is does seem to be Connect/Express specific at the moment.
really - im still getting maintanence prompts, both android app and web browser..
edit, not the connect site - just the commercial website. That's likely to be hosted by another comapny and the websales may also go via another non garmin platform, so not surprised in that case
Yup Garmin site is up but not the connect.
anything I should be doing to protect my details?
With any service that's had an issue like this, I'd change the password. Also, worth checking if they have any second-factor security you can enable.
I was going to get one of their new solar watches today, perhaps I'll hold off.
Having discounted Hammerhead over their lies and messing about the initial backers, together with the lack of sound alerts, I'm now looking at the Karoo 2 with interest as a replacement for my aging Edge 810...
If it was a bad attack they probably took everything offline to protect data/servers from further attacks/sync issues with people trying to upload etc.
A company the size of garmin with the web services they provide, they'll more than likely have multiple server regions, which will all be replicated with each other.
And let's not forget that 'the cloud' is just 'another person's computer' - stuff in 'the cloud' is stored in datacentres all over the world, replicated across multiple servers etc.
If it was a bad attack they probably took everything offline to protect data/servers from further attacks/sync issues with people trying to upload etc.
A supposed company email said exactly that they were everything offline due to a supposed attack at one site.
I've seen cryptolocker attacks happen and they spread very quickly, the first thing you do if you catch it early enough is isolate the affected servers to stop the spread. It may have only started on one server but if that one server replicates data across the world, that's bad times.
It won't come back online until they've patched the security hole, restored the lost data back to the last known good data, and verified all other servers/services across the world haven't been affected. You don't want to be putting stuff back online unless the flaw has been patched and everything is back fully - I'd rather it was done properly than rushed. A few days without garmin data is no loss, really.
They'll have decent backups too, it's the small companies which get hit with these attacks that don't. With decent backups, you can be back up and running the same day - that's for a company with a single datacentre though, not something like garmin which will have infrastructure spread all over the world.
If it was a bad attack they probably took everything offline to protect data/servers from further attacks/sync issues with people trying to upload etc.
A company the size of garmin with the web services they provide, they’ll more than likely have multiple server regions, which will all be replicated with each other.
From reading about the organisation that seem likely to be behind this, it's quite possible that they had thoroughly compromised Garmin's infrastructure, possibly for an extended period, prior to them actually killing stuff. If they had multiple regions (I don't think they do though), then expect them, all to have been compromised. If they could get access to the backups, expect the same. Source code repositories? I'm sure they'd be made a priority.
They'll have taken their time and been thorough, in order to take down as many things as possible and make it as hard as possible for them to restore service without paying money. This won't be a random "computer to computer to computer" infection, but a targeted attack with real people running the show...and it'll be much more damaging as a result.
Well into day 2 now, so it seems that they've been successful to some extent...
It's made it to the BBC - https://www.bbc.co.uk/news/technology-53531178
Just out of interest, I understand the 'Ransom' aspect and the need to pay to decrypt the data but is all the data also compromised and accessible to the hackers for farming and selling on (sorry, I'm a bit vague about all this)?
Just out of interest, I understand the ‘Ransom’ aspect and the need to pay to decrypt the data but is all the data also compromised and accessible to the hackers for farming and selling on (sorry, I’m a bit vague about all this)?
Possibly, it's impossible to know at this stage with the limited information we have. It's plausible that their only goal was to extort money in return for the decryption of files. It would be a much simpler objective than to flog all the personal data but who knows at this stage. If you share a password with any other service you should change it.
Just out of interest, I understand the ‘Ransom’ aspect and the need to pay to decrypt the data but is all the data also compromised and accessible to the hackers for farming and selling on (sorry, I’m a bit vague about all this)?
It's a really good question and the answer, I'm afraid, is "maybe, but probably not". These attacks tend to fall into two camps - the first is where the denial of service is the coercing factor (as is probably the case here) and the second is where revealing data to the public is more critical (e.g. a law firm or bank). While there may be some noise that would come of publicising the world's exercise habits, the reality is that few would really care. After all, most rides on Strava are public anyway (assuming you want to appear on segment boards), so a lot of people wouldn't care.
My guess (and it really is just that) is that this is probably the former type of attack and the greater risk is probably that your data will be destroyed rather than released to the public, but this is pretty unprecedented, so noone really knows...
Generally no, the point of a ransomware attack is to encrypt data so the business can't access it and demand payment to un-encrypt the data. The hackers want the company to know they've been done, often they'll place files giving their demands on the servers.
Data breaches are done to gain access to customer data - email addresses, passwords, etc etc so for that purpose the hackers don't want the target business to know they've been able to get access to the data.
Of course, an attack can be both but generally they're one or the other, they have different goals.
Another type of attack is just done to bring a business to its knees and stop it from operating, which could be a ransomware, or DDOS attack, or other, or all of them.
Another type of attack is just done to bring a business to its knees and stop it from operating, which could be a ransomware, or DDOS attack, or other, or all of them.
I was reading about an example of that sort of thing only this morning, coming from Russia a couple of years ago and affecting a huge swathe of people and businesses, including Maersk.
There's another important part of users' data held by Garmin - our home locations. While you can set privacy zones in Strava and other apps, that only affects what everyone else can see, the data from and to your door is still in the files. If I go for a ride with my wife, my activity shows me starting from our door, hers from the edge of her privacy zone and vice versa.
Simple to write a script to extract the first and last minutes of every activity then offer it to local crims with "bikes at this location" type hints.
It's for that reason I never start tracking near my home.
Oh well, if it wasn't for Covid I'd be somewhere in the wilds of Finland and reliant on my Garmin Inreach for weather forecasts and reassuring the folks at home that I was still alive.
or DDOS attack
Oh oh oh oh, this is what my son expertise is in stopping this.
He has explained it in stupid mum & dad language, but we still have no idea 🤷♀️🤷♂️
Oh oh oh oh, this is what my son expertise is in stopping this.
He has explained it in stupid mum & dad language, but we still have no idea
In very basic terms, it's a flood.
Take that sewer pipe/drain (the server) and chuck so much water (network requests) at it that it can't cope, the drain (server) gets overwhelmed and stops working properly.
Same principle, really.
Web servers (like the one we're posting on right now) can handle a certain number of hit/visits/users at any one time. Pump 100/1,000/10,000x that number and what happens? It can't cope and breaks.