Behind a massive IT failure that grounded flights, upended markets and disrupted corporations around the world is one cybersecurity company: CrowdStrike.
Known as a dominant supplier of software that protects businesses from ransomware attacks, CrowdStrike was thrust into the spotlight on Friday as it struggled to fix a faulty patch that led to cascading, system-wide failures, paralysing the operations of clients ranging from banks to global retail giants to health-care systems.
Wall Street’s reaction to CrowdStrike’s monumental flub was swift: Its shares tanked as much as 15 per cent in New York trading on Friday, wiping out almost $US8 billion of its market value. The company is still worth nearly $US80b ($120b).
CrowdStrike was founded by former executives of antivirus pioneer McAfee and launched in 2012. It has grown into the leading maker of a relatively new type of security software that’s considered among the best defenses against ransomware and other hacking threats. It controls about 18 per cent of the $12.6 billion global market for so-called “modern” endpoint protection software, behind archrival Microsoft’s 25.8 per cent market share, according to market research firm IDC.
The company’s growth has bucked the broader trend in cybersecurity, with the firm reporting blockbuster earnings in its most recent quarter. Chief executive George Kurtz used the opportunity to take potshots at its larger rival, saying in June that CrowdStrike’s technology “creates a wide competitive moat” in the cyber sector, where Microsoft has $US20b in annual revenue.
After a US Department of Homeland Security report faulted Microsoft’s security failings, Mr Kurtz said CrowdStrike had received an “outpouring of requests.”
“There’s a widespread crisis of confidence among security and IT teams within the Microsoft security customer base,” he said.
The type of software CrowdStrike supplies is separate and distinct from older, more limited types of security software.
Traditional antivirus software was useful in the early days of computing and the internet for their ability to hunt for signs of known malware, but it has fallen out of favour as attacks have become more sophisticated.
Now, products known as “endpoint detection and response” software that CrowdStrike develops do far more, continually scanning machines for any signs of suspicious activities and automating a response.
But to do this, these programs have to be given access to inspect the very core of the computers’ operating systems for security defects. This access gives them the ability to disrupt the very systems they are trying to protect. And it is how Microsoft’s Windows systems came into play in Friday’s outage.
Representatives of Texas-based CrowdStrike confirmed online reports a glitchy update was responsible for disabling potentially millions of corporate and government Windows computers around the world and causing the dreaded “blue screen of death.”
The company pegged the incident to “a defect found in a single content update for Windows hosts,” in a statement and said the outage wasn’t due to a cyberattack or security breach. Anyone using a Mac or Linux machine isn’t impacted, the company said, adding that “a fix has been deployed”.
To add to the confusion, an apparently separate incident involving Microsoft’s Azure cloud services also caused disruption on Friday. In a status update, Microsoft said it had fixed the underlying issue but that users would continue to experience “residual impact.”
While cybersecurity professionals say CrowdStrike’s technology is a strong way to defend against ransomware, its cost — which in some cases can be more than $US50 per machine — means that most organisations don’t install it on all of their computers. What that means, however, is that the computers that have the software installed on them are among the most important to protect, and if they go down, key services can fall with them.
Now that CrowdStrike’s fix is out, any Windows desktop or laptop machine with the CrowdStrike product affected by the initial flawed update will need to be updated. The company confirmed in a statement that every affected Windows machine will need to be manually rebooted. In one communication to a customer reviewed by Bloomberg, CrowdStrike’s technical support team advised that rebooting a system up to 15 times may be required.
How laborious that process is will depend on whether CrowdStrike can roll out the fix automatically. In an interview with CNBC on Friday, Mr Kurtz said the company was looking for ways to automate the steps involved, suggesting at least some customers are stuck with manual fixes for now. The company also has addressed the software update at the root of the issue, Mr Kurtz said.
“It was a content bug, or update, that we sent out and we’ve identified, and that we’ve rolled back,” Mr Kurtz said. He apologised to affected customers, and said some systems would take a few hours to come back online while others would take longer.
That means affected entities — from banks to bourses to shipping ports — might take days to fully come back online, according to Alan Woodward, professor of cybersecurity at the University of Surrey.
“To use the laptops, they’ll have to manually intervene – that is a big job,” Professor Woodward said in an interview with Bloomberg News.
There’s also the question of how the bad rollout happened to begin with.
“CrowdStrike is meant to keep these machines safe,” Professor Woodward said.
“This is the sort of thing ransomware would do, but imagine ransomware simultaneously hitting the biggest organisations in the world — container ports in the Baltics, hospitals, railway stations, they’ve all been hit at once because of this one little file.”
CrowdStrike’s customer base comprises large organizations that have a large number of remote machines to manage, he said.
“The economic impact is going to be huge.”