Triple Meltdown: How So Many Researchers Found a 20-Year-Old Chip Flaw At the Same Time

On a cold Sunday early last month in the smaller Austrian metropoli of Graz, three young researchers sat down in front of the computers in their homes and tried to break their most fundamental security protections.

Two days earlier, in their lab at Graz’s University of Technology, Moritz Lipp, Daniel Gruss, and Michael Schwarz had determined to tease out an idea that had nagged at them for weeks, a loose weave in the safeguards underpinning how processors defend the most sensitive recollection of billions of computers. After a Saturday night booze with pals, they got to work the next day, each independently writing code to test a theoretical attack on the suspected vulnerability, sharing their advancement via instant message.

That evening, Gruss notified the other two researchers that he’d succeeded. His code, designed to steal information from the deepest, most protected part of a computer’s operating system, known as the kernel , no longer spat out random characters but what seems to be real data siphoned from the sensitive intestines of his machine: snippets from his web browsing history, text from private email dialogues. More than a sense of achievement, he seemed surprise and dismay.

“It was genuinely, really scary, ” Gruss says. “You don’t expect your private conversations to come out of a programme designed with no permissions at all to access that data.”

From their computers across the city, Lipp and Schwarz soon tested proof-of-concept code they’d written themselves, and could see the same results: Lipp remembers determining URLs and file names materializing out of the digital noise. “Suddenly I could see strings that shouldn’t belong there, ” he says. “I reckoned,’ Oh God, this is really working.'”

Graz University of Technology researchers( from left) Daniel Gruss, Moritz Lipp, and Michael Schwarz represent simply one team of four that independently discovered the same two-decade-old critical security flaw in processors within months of one another.

Graz University of Technology

That night , none of the three Graz researchers slept more than a few hours. The next day, they sent a letter addressed to Intel advising them of a potentially industry-shaking flaw in their microchips. They’d acquired a gap in one of the most basic security defenses computers give: that they isolate untrusted programs from retrieving other procedures on the computer or the deepest layers of the computer’s operating system where its most sensitive secrets are maintained. With their attack, any hacker which is able run code on a target computer could break the isolation around that low-privilege program to access secrets buried in the computer’s kernel like private files, passwords, or cryptographic keys.

On cloud calculating services like Amazon Web Services, where multiple virtual machines coexist in the same physical server, one malicious virtual machine could peer deep into the secrets of its neighbors. The Graz team’s discovery, an attack that would come to be known as Meltdown, demonstrated a critical cracking in one of computing’s most basic safeguards. And perhaps most troubling of all, the feature they had exploited was introduced into Intel microchips in the mid-1 990 s. The attack had somehow remained possible, without any apparent public breakthrough, for decades.

Yet when Intel responded to the trio’s warning–after a long week of silence–the company made them a surprising reply. Though Intel was indeed working on a secure, the Graz team wasn’t the first to tell the chip giant about the vulnerability. In reality, two other research teams had beaten them to it. Counting another, pertained technique that would come to be known as Spectre, Intel told the researchers they were actually the fourth to report the new class of attack, all within a period of just months.

“As far as I can tell it’s a crazy coincidence, ” says Paul Kocher, a well-known security researcher and one of the two people who independently reported the distinct but related Spectre attack to chipmakers. “The two weaves have no commonality, ” he adds. “There’s no reason someone couldn’t help find this years ago instead of today.”

Quadruple Collision

In fact, the bizarre confluence of so many disparate researchers building the same discovery of two-decade-old vulnerabilities raises the question of who else might have found the two attacks before them–and who might have secretly utilized them for snooping, potentially for years, before this week’s revelations and the flood of software fixes from practically every major tech firm that have rushed to contain the threat.

The synchronicity of those processor strike findings, argues security researcher and Harvard Belfer Center fellow Bruce Schneier, represents not just an isolated whodunit but a policy lesson: When intelligence agencies like the NSA discover hackable vulnerabilities and exploit them in secret, they can’t accept those bugs won’t be rediscovered by other hackers in what the security industry calls a “bug collision.”

‘There’s no reason someone couldn’t have found this years ago instead of today.’

Paul Kocher, Cryptography Research

The Meltdown and Spectre incident isn’t, after all, the first time major bugs have been received concurrently. Something–and even Schneier acknowledges it’s not clear what–leads the world’s best security scientists to induce near-simultaneous discoveries, just as Leibniz and Newton simultaneously fabricated calculus in the late 17 th century, and five different technologists independently invented the television within years of one another in the 1920 s.

“It’s weird, right? It’s like there’s something in the sea, ” says Schneier, who last summer co-authored a paper on vulnerability breakthrough. “Something is happening in the middle the community and it results people to reckon, let’s look over here. And then they do. And it definitely occurs lane more frequently than chance.”

So when the NSA receives a so-called zero-day vulnerability–a previously unknown hackable flaw in software or hardware–Schneier highlights the fact that propensity for rediscovery must be free to factor into whether the agency stealthily exploits the glitch for espionage, or instead reports it to whatever party can secure it. Schneier argues bug collisions like Spectre and Meltdown mean they should err on the side of revealing: According to bumpy estimates in respect of the Harvard study he co-authored, as many as one one-third of all zero-days used in a made time may have first been discovered by the NSA.

“If I discover something lying dormant for 10 times, something constructed me discover it, and something more than haphazardly will construct someone else detect it too, ” Schneier says. “If the NSA discovered it, it’s likely some other intelligence agency likely discovered it, too–or at least more likely than random chance.”

Speculative Speculation

While some elements of Meltdown and Spectre’s four-way bug collision–a flaw pile-up may be a better description–remain inexplicable, some of the researchers followed the same public breadcrumbs to their discoveries. Most prominently, security researcher Anders Fogh, a malware analyst for German firm GData, in July wrote on his blog that he had been investigating a curious feature of modern microprocessors called speculative executing. In their insatiable hunger for faster performance, chipmakers have long designed processors to hop-skip ahead in their execution of code, computing ensues out of ordering to save time rather than wait at a certain bottleneck in a process.

Perhaps, Fogh indicated, that out-of-order flexibility could allow malicious code to manipulate a processor to access a portion of remembrance it shouldn’t have access to–like the kernel–before the chip actually checked whether the code should have permission. And even after the processor recognise its mistake and erased the results of that illicit access, the malicious code could trick the processor again into checking its cache, the small part of recollection allotted to the processor to maintain recently used data easily accessible. By watching the timing of those checks, the program could find tracings of the kernel’s secrets.

Fogh failed to build a working attack, due to what other researchers now say were quirks of his testing setup. But Fogh nonetheless warned that speculative executing was likely a “Pandora’s box” for future security research.

Still, Fogh’s post barely voiced alarms for the broader hardware security research community. It was simply months ago that the researchers at the Graz University of Technology started to closely consider his warnings. Their first real clue came instead from the Linux kernel mailing list: In October, they noticed that developers from major corporations including Intel, Amazon, and Google were all abruptly interested in a new defensive redesign of operating systems, called KAISER, that the Graz researchers had created, with the goal of better isolating the memory of programs from the remembrance of the operating system.

The Graz researchers had aimed KAISER to solve a far less serious issue than Meltdown or Spectre; their focus was on disguising the location of a computer’s memory from malicious , not necessarily blocking access to it. “We felt happy, ” Lipp recollects. “People were interested in deploying our countermeasures.”

Soon, however, developers on the mailing list began to note that the KAISER patch could slow down some Intel chips by as much as five to 30 percent for some processes–a far more serious side effect than the Graz researchers had detected. And yet, Intel and other tech monsters were still pushing for the fix.

“There must be something bigger here, ” Lipp remembers belief. Were the tech firms employing KAISER to patch trade secrets, most severe chip-level flaw? Simply then did he and the other Graz researchers think back to Fogh’s failed speculative executing attack. When they decided to try it themselves, the latter are shocked when their somewhat tweaked implementation of Fogh’s technique worked.

They also weren’t alone. Just weeks earlier, by chance, researcher Thomas Prescher at Dresden, Germany security firm Cyberus had finally get around to testing Fogh’s method. “I had looked at it half a year ago and detected the ideas very interesting, but at some degree I just forgot about it.” Prescher says. “In November, I came across it again by chance and only decided to try it. I got it to work very, very quickly.”

In the end, the Cyberus and Graz researchers reported their work to Intel within days of each other in early December. Only after Intel responded to each of the researchers’ glitch reports in the middle of that month did they learn that someone had independently detected and reported their Meltdown attack months prior–as well as the distinct speculative executing attack known as Spectre. That warning came from Project Zero, Google’s elite team of bug-hunting hackers. In reality, Project Zero researcher Jann Horn had seen the two attacks in June–weeks before Anders Fogh’s blog post.

Starting From Zero

How did Horn independently stumble on the notion of attacking speculative executing in Intel’s chips? As he tells it, by reading the manual.

In late April of last year, the 22 -year-old hacker–whose undertaking at Project Zero was his first out of college–was working in Zurich, Switzerland, alongside a coworker, to write a piece of processor-intensive software, one whose behaviour they knew would be very sensitive to the performance of Intel’s microchips. So Horn dived into Intel’s documentation to understand how much of the program Intel’s processors could run out-of-order to speed it up.

He soon insured that for one place in the code he was working on, the speculative execution oddities Intel used to supercharge its microchip hasten could lead to what Horn describes as a “secret” value being accidentally accessed, and then stored in the processor’s cache. “In other terms,[ it would] make it possible for an attacker to figure out the secret, ” Horn writes in an email to WIRED. “I then realized that this could–at least in theory–affect more than just the code snippet we were working on, and decided to look into it.”

‘Something happens in the community and it leads people to reckon, let’s look over here. And then they do.’

Bruce Schneier, Harvard Belfer Center

By early May, Horn had developed that technique into the attack that would come to be known as Spectre. Unlike Meltdown’s more straightforward abuse of the processor, Spectre leverages speculative execution to trick innocent programs or system processes on personal computers into planting their secrets in the processor’s cache, where they could then be leaked out to a hacker performing a Meltdown-like timing attack. A web browser, for instance, could be manipulated into leaking a user’s browsing history or passwords.

Spectre is harder for attackers to exploit than Meltdown, but also far more complex to set. It likewise operates not only in Intel microchips, but across ARM and AMD microchips too, an even thornier and longer-term difficulty for the industry. Horn reported his findings to the chipmakers on June 1. And as he continued to explore speculative execution’s other prospects, he found and reported the Meltdown attack to Intel three weeks later.

Finally, there would be one more coincidence in the storm of flaw collisions around Meltdown and Spectre. Just around the time that Horn was beginning to test his attacks, Paul Kocher was starting a sabbatical from the San Francisco-based company he’d founded, Cryptography Research. He wanted day, in part, to explore a broad issue he saw in computer security: the increasingly desperate drive to squeeze ever-greater performance out of microchips at all costs–including, perhaps, the cost of their fundamental security.

At a cryptography and hardware meeting in Taipei last September, Kocher’s former colleague Mike Hamburg raised mistrusts about speculative execution. Kocher was immediately determined to prove their own problems. “It wasn’t so much of an’ aha’ moment as an an’ eww’ minute, ” Kocher says of the realization that produced him to the same assault technique. “As soon as I started to look at speculative executing, it was pretty clear to me as a security person that this as a very bad idea.”

Not long after he’d returned from Taipei, Kocher had coded a working exploit of his own–with no knowledge that Google’s Horn had found exactly the same decades-old issue only months earlier.

Outlier or Telling Anecdote?

For Kocher, the key question isn’t how so many researchers stumbled onto the same class of strike at roughly the same period. It’s how the attacks remained undiscovered for so long–or whether they were in fact detected, and used to hack unwitting targets in secret.

“If you asked me whether intelligence agencies acquired this years ago, I would guess certainly yes, ” Kocher says. “They have some of the world’s best efforts at these sorts of things. It would be quite likely they would have noticed. And if they found something like this, as long it’s yielding good intelligence, they don’t tell anyone.”

“It’s not just the NSA, ” he adds. Other state-sponsored hackers likely have the skills–and had the time–to have potentially saw the Spectre and Meltdown attacks, too.

On Friday, White House cybersecurity coordinator Rob Joyce, a former senior NSA official, told The Washington Post that the NSA didn’t know about Spectre and Meltdown and “ve never” exploited the flaws. Joyce has also touted a move to disclose more about the NSA’s rules for disclosing vulnerabilities it find, a policy known known as the Vulnerabilities Equities Process.

‘If you asked me whether intelligence agencies detected this years ago, I would guess surely yes.’

Paul Kocher

Despite the nearly uncanny anecdotal evidence for bug rediscovery that Spectre and Meltdown represent, it’s far away from clear just how common that phenomenon has become. The Harvard Study co-authored by Bruce Schneier, for one, investigated a trove of bug report data containing 4,300 vulnerabilities. Fourteen percent of Android vulnerabilities were reported again within merely 60 periods of their initial breakthrough, and around 13 percentage of Chrome flaws. “For the NSA, holding onto vulnerabilities is style more dangerous than you’d think, given the raw numbers, ” Schneier says.

But another study liberated last year by the RAND corporation, which looked at flaws from an unnamed research organisation, found only a 5.7 percentage possibility that a dedicated glitch would be found again and reported within a year–although such studies didn’t account for other, secret glitch discoveries.

Lillian Ablon, one of the RAND study’s writers, visualizes the Spectre and Meltdown rediscoveries not as a broad sign that all bugs are found several times over, but that trends in computer security can abruptly concentrate many eyes on a single, narrow battleground. “There may be bug collisions in one area, but we can’t attain the grand statement that glitch crashes happen all the time, ” she says. “There will be codebases and classes of glitches where no attention exists.”

Paul Kocher argues the real lesson, then, is for the security research community not to follow in each others’ footsteps but to discover and fix glitches in the obscure code that rarely attracts widespread attention.

“Throughout my career, whenever I’ve looked somewhere there isn’t a security person seeming, I find something nasty and unpleasant there, ” Kocher says. “The shocker for me is that these attacks weren’t detected long ago. And the question that I struggle with and fear is, how many other things like this have been sitting around for 10 or 15 years? “

More Meltdown

Meltdown and Spectre are as devastating as they are complicated. Here’s how they operate, and why they’re such a danger.

Fortunately, some important steps have already been taken to fix the problem–but a full answer is still years left.

This is also the latest in a string of bumpy security mistakes for Intel, including a recent, critical vulnerability in its Management Engine.

What do you think?

0 points
Upvote Downvote

Total votes: 0

Upvotes: 0

Upvotes percentage: 0.000000%

Downvotes: 0

Downvotes percentage: 0.000000%

Scientists Discover Clean Water Ice Just Below Mars’ Surface

James Damore’s Lawsuit Is Designed to Embarrass Google