How the CrowdStrike outage carved out new opportunities for hackers

In the wake of a scathing U.S. government report that faulted Microsoft for having a security culture that let Chinese hackers access the inboxes of top federal officials last year, CrowdStrike used the findings as leverage to promote its own cybersecurity services as safer alternatives.

“Considering Microsoft? It’s your adversaries’ favorite target,” says a web page advertising its offerings that garnered attention in the days following the release of the report from the Department of Homeland Security’s Cyber Safety Review Board. “Microsoft’s security products can’t even protect Microsoft. How can they protect you?” it adds.

The bold claims seemingly backfired when a faulty patch rolled out by CrowdStrike early Friday morning inadvertently crippled Microsoft-run devices around the world. The outages, which are still being remedied, hit as many as 8.5 million computers, Microsoft said in a Saturday blog post.

Compared to last year’s Chinese email hack — which targeted select federal government Microsoft Exchange inboxes — Friday’s outages were wider-ranging, more impactful and had real-time consequences that experts say have laid groundwork for future hacking schemes and scarred CrowdStrike’s reputation as a star player in the eyes of federal regulators.

“[The incident] demonstrates that even gold standard cybersecurity solutions in the market need to be cautious about how they frame what they’re capable of doing, based on the fact that, on any given day, something could go terribly wrong,” said Chris Cummiskey, a former DHS official who also served as chief information security officer for the State of Arizona.

“Every company I talked to was affected in some way.”

CrowdStrike is regarded as a top cybersecurity services firm, famous for its endpoint protection products designed to stop digital adversaries from infiltrating networks by shielding “endpoint” devices like laptop computers or phones that often provide hackers an entryway into systems.

The company’s flagship Falcon platform is used by hundreds of global organizations, as well nearly all U.S. states, according to its website. In order for Falcon to deter hackers from accessing a client’s systems, the product needs to tether onto devices at a deep level. Once installed, CrowdStrike has full access to the crown jewels of a customer’s computers, where it can stop threats moving about at all points inside a device.

But Falcon, like any piece of software, requires periodic updates. The patches, which are developed by CrowdStrike’s product staff and deployed remotely to their clients, often involve changes to Falcon’s contents that instinctively interact with a computer’s core operating functions.

In normal circumstances, the changes undergo rigorous quality assurance testing before rolling out to devices. For many security practitioners, it’s a mystery as to how such a powerful company released a seemingly half-baked patch with the massive consequences that followed.

“Shit happens. The thing is that, usually, when shit happens, it doesn’t get pushed to production,” said a cybersecurity consultant that was in contact with over a dozen companies affected in the Friday outage. The source requested anonymity to speak openly about his experience during the incident.

The recovery process will vary at each company, he said, estimating that a full, back-to-normal reset could take weeks or months. Some organizations’ IT staff will have to manually reboot machines one-by-one, sometimes through a single USB drive, the consultant said. 

In some cases, companies will have to deal with an extra obstacle enabled by BitLocker — a Windows security tool that encrypts volumes of systems’ data to protect against unauthorized access — that’s stopping IT admins from lowering into their computers’ operating systems to extirpate the faulty update.

“It’s a ‘break glass, go fix everything’ scenario,” he said. “Every company I talked to was affected in some way.” 

Hacker’s haven

The outage has already created secondary hacking opportunities being leveraged by cybercriminals. CrowdStrike and the Cybersecurity and Infrastructure Security Agency in DHS warned in a Saturday blog that hackers targeting Latin American customers are being sent sham messages with a folder dubbed “crowdstrike-hotfix.zip” that claims to clear out the contents that enabled the outages.

In reality, the code, if executed, lets hackers infiltrate a victim’s machine by secretly injecting malicious code into the core functions of Windows applications. This allows them to stealthily infect the machine with malware that can be used to sabotage the targeted computer.

But the scale may be greater than just those phishing scams, said Silas Cutler, an independent security researcher who spent seven years on CrowdStrike’s threat intelligence team.

“Whenever there is a large incident, communication is always the most important thing,” he said, stressing that there are now opportunities to inject fear into peoples’ minds about using endpoint tools that, on most occasions, have stopped major cyberattacks.

“It’s trying to take advantage of people’s fear to get them to lower their security further than what they’d have without something like CrowdStrike,” he said. “I think the real risk is going to be people carrying [endpoint] systems out and jumping to something else out of panic.”

The high degree of public attention on the issue also became a great mapping exercise for hackers, said Cummiskey, the former DHS official. “You don’t want that kind of visibility for bad actors to be able to see what companies are using or how they’re configuring their security” because it gives adversaries another pathway into company and government tech stacks, he said.

Airlines, banks, and hospitals were just some of the services impacted by the outage, according to multiple reports. As of Sunday, airports are still log-jammed as they work to get systems back online.

CrowdStrike will likely survive and move forward, but, reputationally, it can’t afford another incident like this, said William MacMillan, a former CISO at the CIA.

“The update was supposed to be seamless to users,” said MacMillan, now chief product officer at cybersecurity firm Andesite. “Endpoint detection and response has been game changing, but because of where [Falcon] is positioned on the endpoints, if a patch does go wrong … that can have very significant consequences, as we’re seeing across the globe.”

When the feds come knocking

The incident soon prompted a statement from CISA Director Jen Easterly, who called the outages a “serious mistake” in a LinkedIn blog that also applauded CrowdStrike and its CEO George Kurtz for being “transparent, responsive, and professional” with the agency.

CISA itself was impacted, according to an analyst who spoke to Nextgov/FCW on the condition of anonymity because they were not permitted to provide updates on the internal status of the agency’s systems.

Several other federal entities said they were affected, including the Social Security Administration, Treasury Department, Customs & Border Protection and the GSA-managed Login platform used to verify government employees when they log into their workstations, which is often susceptible to outages at upstream providers. President Joe Biden was also briefed, the White House said.

Cummiskey, who now runs his own consulting practice, said he spoke with several government IT officials over Friday and Saturday who expressed the longstanding concern of “single points of failure” in a technology nexus, where one flaw, even if unintended, can shutter an entire organization. Lawmakers are likely mulling oversight hearings about the incident, he added.

Kurtz said in an X post that the company is “working on a technical update and root cause analysis” that will be shared publicly. A company spokesperson flagged a technical blog that explains the incident in detail.

The full extent of the outage’s impact on federal government operations is still not known. Crowdstrike is in wide use across federal agencies and it is a key vendor on the governmentwide Continuous Diagnostics and Mitigation cybersecurity support services contract. The company has also secured contracts with the Justice Department and State Department, according to GovTribe, a federal market intelligence platform owned by Nextgov/FCW parent company GovExec.

A senior administration official said Friday the White House has been convening agencies to assess impacts to the U.S. government’s operations and entities around the country.

For now, CrowdStrike may have to rethink how it brands itself, especially when it calls out other competitors in the way it did following the DHS-Microsoft findings on last year’s email hack, said MacMillan, the Andesite CPO.

“I don’t think that type of language is particularly helpful. I think the entire industry should think about long term incentives and long term reputations for building products that are … maintained well,” MacMillan said. “Let’s all spend less time leveraging bad things that inevitably happen to try to smear one another.”