The Kaseya Ransomware Attack is a Really Big Deal
By Matt Tait Monday, July 5, 2021

LAWFARE

A keyboard (ericnvntr/https://flic.kr/p/9A8uFb/CC BY 2.0/https://creativecommons.org/licenses/by/2.0/)

If you’re not already paying attention to the Kaseya ransomware incident, you should be. It’s likely the most important cybersecurity event of the year. Bigger than the Exchange hacks by China in January. Bigger than the Colonial pipeline ransomware incident. And, yes, more important than the SolarWinds intrusions last year.

First, some background. Kaseya is a managed service provider; its customers use Kaseya to manage their company IT infrastructure. As part of this task, Kaseya can deploy software to the systems under management, in a way that is broadly equivalent to a software provider deploying an automatic update to those machines. For those interested in more, Nick Weaver wrote a piece for Lawfare that walks through the background in depth.

Under normal circumstances, automatic software deployment, especially in the context of software updates, are a good thing. But here this feature was turned on its head. Russian-based criminal gang REvil hacked into Kaseya’s management system, and pushed REvil software to all of the systems under Kaseya’s management. From there, the ransomware promptly disabled those computers and demanded a cryptocurrency payment of about $45k per system to set the machines free. As of writing, REvil claims that about a million total computers were affected, and is offering a “bulk discount” of $70m to unlock all affected systems in a single payment.

Although the direct impact is already enormous, to me, the direct impact is, in some sense, far less important than the issue of how the incident occurred, namely by subverting software delivery mechanisms as a means to install ransomware.

Supply chain attacks such as these are the proximate technical cause of many of cybersecurity’s “greatest” hits, including NotPetya and SolarWinds. The NotPetya attack in June 2017 did about $10bn or so of damage globally. The SolarWinds campaign led to the compromise of thousands of major organizations and dozens of federal agencies. NotPetya was delivered by a malicious update to Ukrainian accountancy software firm MeDoc; the SolarWinds malware by a malicious update to SolarWinds’ IT management software.

If this is not yet enough to catch your attention, three further observations will.

First, supply chain compromises, such as these, are very often indiscriminate; everyone that installs a malicious update gets the malware. Even in cases where supply-chain malware merely lays the groundwork for further sub-targeting after the initial breach—as the SolarWinds malware did—the effect is disruptive to all recipients, whether sub-targeted or not. Except in very rare cases, perpetrators behind supply chain attacks cannot control, predict or constrain the real-world consequences of subverting software supply-chains—and this is especially true when they are used to install ransomware.

The second, and perhaps scariest, observation is that the software vendors used in malicious update compromises thus far have, in the grand scheme of things, been relatively small. MEDoc, SolarWinds and Kaseya are, of course, important to their respective customers, but none were household names before their respective incidents. Far bigger software vendors exist. Some are central to the basic functioning of modern computing. A disruption to the supply chain of platform vendors like Microsoft, Apple, or Google would have fallout at a scale that is literally unimaginable; with global disruption so vast that it cannot really be articulated without sounding insane or alarmist. But platform vendors are not the only large software developers. Hundreds of smaller companies exist on the periphery, each with enormous customer-bases, from organizations like NVidia, Dell, Adobe and Mozilla; Linux and its various major distributions; the maintainers of core package managers used by huge numbers of software developers; large enterprise IT products; as well as any of the major games companies like Blizzard Activision or Valve. Most of these regularly push software to huge numbers of users and organizations at an operational scale that makes MEDoc, Kaseya and SolarWinds look like lightweights.

The final observation is that defensive remediation of ransomware deployed through automatic updates is pathological to the cybersecurity industry itself in a way that is qualitatively different from other categories of cybersecurity incidents.

To understand why, contrast the Kaseya breach with, say, a more “traditional” deployment of malware using zero-day exploits against each affected target. Hackers who develop or have access to zero-days have two natural obstacles to their mass-use: the difficulty in reaching exposed systems, and the risk of discovery. Once the zero-day is discovered, its utility rapidly and sharply declines.

Once a zero-day is discovered being used “in the wild”, remediation typically comes in the form of two key streams in the cybersecurity community. The first stream is the software developers at the affected vendor. Those developers quickly reverse-engineer the exploit to identify the software defect. For simple fixes, the developers can fix it outright; for more complicated issues they might temporarily disable the surrounding feature until the defective component can be safely re-engineered. In either case, an “immunized” patch is made ready and deployed to customers, often within a few days of initial discovery of the exploit.

As the developers engineer a patch to prevent new infections, the incident response community mobilizes to help infected organizations. These incident responders perform an intensive triage and remediation. They identify and restore affected systems, discover what was stolen, and put in place measures that will make future compromises less likely to occur—and less damaging when they do—for the affected systems and organizations.

Malware deployed automatically via the supply-chain up-ends all of these dynamics pathologically. A malware operator with access to automatic software delivery infrastructure has no incentive to keep the infections small. Rather than infecting only a few targets at the top of their priority list, the hacker typically hacks all affected customers nearly simultaneously. Finding and reaching exposed systems isn’t an obstacle here to their mass-deployment either; the delivery mechanism “helpfully” routes the malware through to systems buried deep inside corporate networks, or hidden behind layers of traditional defenses.

The vendor can’t respond in the normal way to supply-chain malware either. The malware came from their own software delivery infrastructure, so remediation begins with them disabling their infrastructure to prevent further misuse and then turning inwards to secure their own systems. In any case, patches are the wrong tool for remediation: patches help defend systems that might be vulnerable to malware, but here their customers are already infected with the malware. By the time the breach is discovered, it’s already too late to fix via a patch.

As if this wasn’t enough, malware-laden updates are also pathological to incident response. Since malicious updates affect enormous groups of systems simultaneously, they tend to saturate the capacity of the entire incident response industry overnight, overwhelming their ability to respond.

In short, software supply chain security breaches don’t look like other categories of breaches. A lot of this comes down to the central conundrum of system security: it’s not possible to defend the edges of a system without centralization so that we can pool defensive resources. But this same centralization concentrates offensive action against a few single points of failure that, if breached, cause all of the edges to fall at once. And the more edges that central failure point controls, the more likely the collateral real-world consequences of any breach, but especially a ransomware breach will be catastrophic, and cause overwhelm the defensive cybersecurity industry’s ability to respond.

Tackling this problem is no small task; it will need a great deal of resources and creativity across a large number of different domains, from the technical community through to the foreign policy community. And, to be fair, many of the options towards a safer infrastructure will likely require some large, and frankly unpopular, shaping-up against some large entrenched interests to make progress.

But before researchers and policymakers can start to look for solutions, the first step is recognizing why supply-chain compromise is fundamentally different from most other problems encountered day-to-day in cybersecurity, and one with a failure mode that can be unusually fast and large-scale. Only then will the information security community be able to start tackling it with the scale and seriousness that it deserves.