Just hours before the Fourth of July weekend, a huge, coordinated cyberattack hit hundreds of businesses across the world. A group of hackers broke in by exploiting a hole in the software code of an information technology company with a wide-ranging client base, then demanded $70 million in ransom.

A large grocery chain in Sweden shut down hundreds of stores. Schools in New Zealand warned that staff might not be able to use computers.

It was the biggest case yet of a scourge that affects the world’s companies and government agencies nearly every day, and that is only getting worse. In an analysis of publicly reported ransomware attacks against health care providers, municipalities and schools, The Washington Post found that ransomware attacks in the United States more than doubled from 2019 to 2020.

To reconstruct the anatomy of a ransomware attack, The Post conducted its own data analysis and spoke with nearly a dozen cybersecurity experts, law enforcement officials, negotiators and victims. The Post used different examples to illustrate the components of how an attack happens. The resulting examination has five parts: the hackers, the hack, the negotiation, the payment and the aftermath.

The costs of such attacks add up. Some experts conservatively estimate that hackers received $412 million in ransom payments last year.

“Criminals are economically motivated, and they are not dumb,” said Joshua Motta, chief executive of cyber-insurance company Coalition. “For a while, it was credit-card skimming, data breaches, reselling Social Security numbers. Ransomware is a considerably more lucrative business model.”

A ransomware attack shut down IT systems and caused major disruption for Ireland’s public health service in May. The Health Service Executive identified the attackers as one of the largest cybercriminal gangs operating today, known as Conti.

The organization, which researchers believe is based in Russia, has an unknown number of hackers working together in a hierarchical structure, operating almost like a legitimate business. It has developed a malware that can crawl through computer systems and lock down files, and it employs representatives to communicate with victims, said Rick Holland, chief information security officer for cyberthreat software company Digital Shadows.

Some attacks, like the one on Kaseya, happen when hackers find a vulnerability in a company’s software and use that to get into their system. But most use relatively unsophisticated methods to break into computers, such as sending phishing emails that trick employees into opening an attachment or clicking on a link that downloads malicious software, which goes on to encrypt files and bar access to the whole network.

Conti then sends a message that demands a ransom in exchange for a “decryption key,” or a computer program that will unlock the files. Companies or negotiators can message with Conti hackers to get a discount or make a deal, as well as work through logistics such as where and when the money will be sent.

“If you ignore them, you can tick them off and then they may start going to media, they may start going to social media, they may start going to blog sites,” Holland said.

Conti also threatens to leak stolen company information.

“If you are a client who declined the deal and did not find your data on cartel’s website or did not find valuable files, this does not mean that we forgot about you, it only means that data was sold and only therefore it did not publish in free access!” Conti says on its website, available only on the dark web, which requires special software to access.

Attempts to reach Conti for comment failed.

It has leaked information stolen from more than 418 organizations since it was established in January 2020, according to cyber firm Recorded Future. The actual number of companies it has hacked is likely to be orders of magnitude larger. Conti was cited in May by the FBI for hacking 16 health care and first-responder organizations across the United States.

Ireland’s Health Service Executive refused to pay the ransom, but Conti relented and handed over the keys to unlock the systems to “lessen criticism,” according to the Irish Times. Still, health care services were “severely impacted” by late June, the service said in a news release. A spokesperson for the service confirmed it had not paid a ransom and pointed to a comment its chief executive made last month.

“We did get a decryption key. ... It’s not any kind of simple off-the-shelf software,” CEO Paul Reid said at a mid-June news conference. “It’s a very clunky, unsafe set of locking codes that we had to make safe.”

And Conti is just one such group. Many attacks come from organized groups that operate with relative impunity out of Russia, Belarus and other East European countries, according to researchers. Attackers range from enterprising individuals all the way up to groups of hundreds working directly for a nation state like North Korea.

The highest-profile attacks are often conducted by hacking groups that researchers say operate somewhat like regular companies, with employees, revenue goals and internal hierarchies.

Many of these groups, like DarkSide and REvil, offer “ransomware as a service,” selling their malware to whoever has the know-how to execute a hack and has a target.

- - -

Just before 5 a.m. on May 7, a Colonial Pipeline employee saw a ransom note on a computer screen. Many of the IT or business-side computers in the company’s network had been locked up — their data encrypted — by hackers. And they wanted money to open them back up.

Colonial Pipeline’s experience was unique in that it led the company, as a precaution, to shut down its entire pipeline. But it’s not just big companies that get hit. For the tens of thousands of smaller American companies that have been targeted by ransomware criminals over the last several years, the feeling of seeing a ransom note is all too familiar.

“They’re absolutely terrified,” said Keith Swanson, director of counterextortion and threat intelligence at Kivu Consulting, which helps victim companies negotiate with hackers. “They don’t know what to do. They’ve never dealt with anything like this in their lives.”

Roughly one-third of American companies have cyber insurance, although it’s getting harder and more expensive to obtain as attacks surge. Generally, the insurer acts as a hub to help the victim with everything from investigating the attack to remediating compromised systems, negotiating ransoms and navigating legal and public relations issues.

A first call goes to a cybersecurity company that can investigate, help contain damage and repair systems. Colonial, for instance, confirmed that it is working with Mandiant, a leading cybersecurity firm now owned by FireEye. It responds to more than 1,000 breach incidents a year — a little more than one-third of them involving some form of ransomware, financial extortion or payment card theft.

“The first thing we do is try to assess what’s going on,” said Charles Carmakal, Mandiant’s chief technology officer. That includes how hackers got in, if they still have access, what they did and what data was exposed.

To answer the questions, Mandiant needs access to network data. It remotely deploys a software forensics tool that it uses to reach into every single computer in the victim’s network.

Firms like Mandiant also help get networks back online. That might entail rebuilding systems that were infected with malicious software, decrypting files encrypted by the hacker and improving defenses to fend off further attacks.

The process can take days to weeks for an organization that hasn’t been hit too hard and has good backups of its files, Carmakal said. For larger organizations, or those without good backups, the process can take months.

Often, the ransomware actors don’t stop at simply encrypting data. They steal it, too. And they go after the most sensitive data they can find — tax records, business negotiations and intellectual property.

If a victim can avoid paying a ransom, and restore its network from backups, that’s great, experts said. But, said Kivu’s Swanson, a “lot of times the networks are nonoperational — completely fried. The bad guys will come in and delete the backups or encrypt them.”

In that case, the victim either has to pay the ransom or rebuild its entire network from scratch. Even if the victim pays and receives a decryptor — software that returns encrypted data to its original state — the process of restoration can take anywhere from a few days to weeks, said Austin Berglas, global head of professional services for the cybersecurity firm BlueVoyant.

The entire experience is grueling. “Even if a company has a solid incident-response plan and they’ve practiced it, it’s still massive panic,” Berglas said. “If their business is shut down, it’s all hands on deck. Nobody’s resting. It’s full-force.”

Kurtis Minder, founder of the cybersecurity service GroupSense, recently helped a cancer charity that was locked out of its computer network negotiate down a ransomware payment.

“We were pretty direct with the bad guy. Like, ‘Look, shame on you, dude. They don’t have it. They use it to prevent breast cancer,’ ” Minder said.

That tactic worked in bringing the ransom down, he said. “We didn’t get it for zero, but we got it for almost zero.”

When a company is hacked, the attackers will generally leave a ransom note. The note can be as simple as an email to the company’s executives or a text file left unencrypted on one of the servers. Sometimes hundreds of computers in a room might all be swept with red banners that say “You have been infected,” said James Turgal, a former FBI agent who’s now a vice president at cybersecurity firm Optiv.

The note typically contains instructions on how to access a website on the dark web. That’s where hackers will say how much they want, and how much time the victim has to pay up. A countdown clock sometimes ticks away, giving a company a set amount of time, usually about a week, before the price goes up.

Those who negotiate often hire a professional. Minder founded GroupSense in 2014 to study criminal hackers and help other companies protect against them. His analysts, who together speak more than a dozen languages, scour the dark web to learn about hacking groups and then notify companies that they might be compromised.

“We do [negotiations] virtually every day, sometimes multiple times a day,” Minder said. “We’ve got lots of information about these bad guys. We know where they operate. We know a lot of things about them.”

He has gotten requests for help from victims including flower shops, print shops and microbreweries.

For large companies, hackers usually ask for a sum of money based on information dug up from stolen financial statements. For smaller companies, the opening demand can be more arbitrary, Minder said.

In those cases, hackers may ask companies for relatively low amounts, like $10,000, to encourage speed, said Berglas. More-sophisticated hackers hitting larger companies often ask for initial payments in the millions of dollars.

Negotiations are usually in English, but it’s often clear that the messages are being run through Google Translate because the hacker’s language doesn’t appear to be that of a native speaker. The tone of negotiations is generally civil and businesslike, Minder said. “They’re perpetrating this as if it’s normal business and that they’re doing some sort of security favor for the victim.”

The hackers expect the negotiator to try to bring down the price. “It’s almost like used-car dealers,” Minder said. “They know you’re not paying the price on the sticker. And that’s why the price on the sticker is what it is.” For small companies, the entire process takes two to four days. For bigger ones, it could go as long as three weeks.

Data on what companies actually pay to hackers is spotty, because many victims don’t disclose the attacks in the first place. The average payment in the fourth quarter of 2020 was $154,000, down from a peak of over $230,000 earlier in the year, according to cybersecurity consulting firm Coveware.

Generally, Minder’s firm has been able to get the ultimate amount companies pay down to between 10 and 40 percent of the original demand, he said.

- - -

JBS, the world’s largest meat processor, was hacked in late May, a system breach that caused it to shut down beef plants across the United States and disrupted operations for days. On June 1, it paid an $11 million ransom to prevent customer data from being compromised.

“This was a very difficult decision to make for our company and for me personally,” JBS USA chief executive Andre Nogueira said in a statement at the time.

Tom Robinson, a blockchain researcher and co-founder of the analysis company Elliptic, said he scrutinized the digital ledger, a publicly available online list of all transactions made with bitcoin.

He was able to track the bitcoin ransom payment from JBS to a “wallet” he believed is associated with a cybercriminal gang thought to be based in Russia.

Bitcoin is a cryptocurrency, a digital form of money that is created through massive amounts of computing power and can be traded virtually through a series of private “wallets” and public “exchanges.” Exchanges are organized markets, managed by companies, where people bring their cryptocurrencies to switch them into dollars, pounds or other forms of money — they operate somewhat like currency exchanges at banks.

In most cryptocurrencies, every transaction is recorded on the ledger, known as the blockchain. That’s how Robinson could see JBS’s 301 bitcoin payment get sourced on a U.S. exchange and transferred from its hands to a private wallet, presumably belonging to the cybercriminals.

Within hours, the funds were scattered into hundreds of wallets.

“Cryptocurrency is invading all forms of criminal activities, and criminals follow the money,” said Gurvais Grigg, a former assistant director of the FBI and now global public sector chief technology officer of blockchain analysis firm Chainalysis.

The transactions are also irreversible, said Rich Sanders, co-founder and lead investigator at CipherBlade, which analyzes the blockchain. “You can’t go to complain to bitcoin and ask for a charge back.”

Some of the JBS funds were sent through digital “mixers,” which operate as a digital form of money laundering. Mixers use software to commingle and swap one bitcoin for another, all with the purpose of breaking the chain so the history of a single coin is more difficult to trace.

Hackers tend to get caught when they want to exchange their digital currency for traditional cash, experts said.

Investigators try to identify and label funds on the blockchain to keep track of them. If the money is ever moved from a private wallet to a public exchange, the researchers or law enforcement can directly contact the exchange operators and ask them to lock the account in question while they investigate.

Many exchanges based in the United States will cooperate with these requests, said Megan Stifel, senior policy analyst at the Global Cyber Alliance. They comply with common financial regulations, such as “know your customer,” meaning they have identification for account holders.

But there are exchanges that purposely ignore or try to resist requests or are based in jurisdictions that have lax regulations or look the other way.

Federal authorities did manage to recover more than $2 million worth of Colonial’s $4.3 million ransom payment. In that case, researchers say, officials seem to have accessed a private wallet that contained the cryptocurrency.

Private wallets are difficult to access because they require an encryption key, a long string of numbers and letters, that only the wallet holder possesses. It’s unclear how officials got hold of the wallet’s key.

- - -

It takes the better part of a year — an average of 287 days — for a company to fully recover from a ransomware attack, according to a wide-ranging April report from a group of more than 60 experts from industry, government, nonprofits and academia known as the Ransomware Task Force.

For many companies, the actual ransom payment isn’t even the most expensive part of the attack. Companies have to restore backups, rebuild systems, work with forensic investigators to ensure that the hackers are truly locked out and, in many cases, implement stronger cybersecurity controls to prevent future attacks.

And the effects of an attack can reach well beyond the company’s own doors, affecting people’s everyday lives and crucial services. Ferry services were disrupted on the East Coast when a breach brought down a ticketing system. JBS meat-processing plants temporarily shuttered some operations when an attack hit the company’s systems.

Governments around the world are trying to find ways to crack down, with the Group of Seven industrialized countries committing to work together to thwart the attacks, and the White House saying it’s “not taking any options off the table” when it comes to possible responses to the Russian government being slow to stem the tide of attacks originating from inside the country.

Various U.S. agencies began earlier this year to launch ransomware initiatives. The Department of Homeland Security’s cybersecurity agency in January kicked off a campaign to prod public and private-sector organizations to adopt measures to reduce their risk of being victimized by ransomware. Last fall 2019, that agency launched a similar initiative to encourage state and local officials to secure election infrastructure against ransomware attacks. In April, the Justice Department created a task force to disrupt the criminal ecosystem that fuels ransomware attacks.

But the Colonial Pipeline attack fueled a more concerted effort as the White House with President President Biden launching an initiative to address the dangers of ransomware. The initiative complements an executive order he signed in May to shore up the federal government’s digital defenses, which officials hope will spur the private sector to bolster its own.

During a June 16 summit in Geneva, Biden and Russian President Vladimir Putin discussed cyberattacks and agreed that the two countries would begin strategic talks on cybersecurity.

In the meantime, cybercriminals are doubling down on ransomware attacks, having proven to themselves and other potential hackers that they can be extremely profitable.

“You’ve got a lot of folks out there, bad threat actors, that are now emboldened and certainly motivated by the high numbers that are reported on the ransomware payments,” Turgal, the cybersecurity expert from Optiv said. “It’s not going to go away.”

The Washington Post’s Dalton Bennett and Zach Levitt contributed to this report.

related stories

• Pressure grows on Biden to curb ransomware attacks