Dan Kaminsky: Finding a DNS Flaw and Organizing the Fix

In early 2008, Dan Kaminsky found a bug that could've let anyone hijack any website on earth, and then did something stranger than finding it: he persuaded the whole industry to fix it before he'd say what it was.

He was 29, a penetration tester from San Francisco who gave the most crowded talks at Black Hat and once counted how many networks Sony had secretly infected with a rootkit. Later he'd hold one of the seven keys that can rebuild the internet's address book.

The New York Times called him a digital Paul Revere. His friends called him the guy who kept the lights on, and in 2021 the lights went out far too early.

Dan Kaminsky at a Glance

• Born February 7, 1979, in San Francisco, Kaminsky got a Radio Shack computer at four, was coding by five, and at 11 drew an angry phone call from a military network administrator.
• He studied at Santa Clara University and worked for Cisco and Avaya before becoming director of penetration testing at the Seattle firm IOActive.
• In 2005 he used a technique called DNS cache snooping to show that Sony BMG's copy-protection rootkit had reached more than 568,000 networks.
• In 2008 he discovered a fundamental flaw in the Domain Name System, quietly convened the world's DNS vendors at Microsoft, and shipped a coordinated patch on July 8, 2008.
• In 2009 he and two Honeynet Project researchers worked out how to detect machines infected by the Conficker worm.
• In 2010 ICANN made him one of seven Recovery Key Share Holders for the DNSSEC root, the people who could restore the system's master key.
• In 2012 he co-founded the anti-fraud company White Ops, now HUMAN, as chief scientist.
• He died at his home in San Francisco on April 23, 2021, aged 42, and was inducted into the Internet Hall of Fame later that year.

The Life of Dan Kaminsky

Kaminsky's Early Years

Daniel Kaminsky grew up in San Francisco with a Radio Shack computer his father gave him at four. By five he'd taught himself to code, and by 11 he was poking around military networks he had no business seeing.

The call came to his mother, Trudy Maurer: an administrator wanted the family's internet cut off. She told him that if he did, she'd take out an ad in the San Francisco Chronicle announcing that the military's security was so poor that an 11-year-old could break it.

They settled on three days without the internet. It's the only punishment the record shows Kaminsky ever receiving for his curiosity.

From Cisco to IOActive

Kaminsky studied at Santa Clara University, interning at Cisco along the way and contributing to the security book Hack Proofing Your Network before he graduated. Jobs at Cisco and Avaya followed, and then a long run at IOActive, where he became director of penetration testing.

He was already a conference fixture. His "Black Ops of TCP/IP" talks at DEF CON and Black Hat were part lecture and part magic show, built around tricks for making the plumbing of the internet do things it was never meant to do.

Counting Sony's Rootkit

In 2005 the music giant Sony BMG was caught shipping CDs that silently installed a rootkit on Windows PCs. Everyone knew it was bad, but nobody knew how widespread it was.

Kaminsky worked out a way to measure it. The rootkit phoned home to a handful of Sony domains, and any DNS server that'd looked those names up would remember them for a while, so he simply asked more than three million DNS servers whether they knew the addresses.

Some 568,200 did, which meant the rootkit had reached at least that many networks around the world. The Electronic Frontier Foundation spread the number the same week, and the technique, DNS cache snooping, became a Kaminsky trademark.

The Bug That Could Break the Internet

Early in 2008, Kaminsky stumbled on something far bigger. The Domain Name System turns names into addresses, and he found a way to forge its answers so reliably that an attacker could redirect an entire domain's traffic in seconds.

Every bank login, every software update and every email could be sent somewhere else. Worse, the flaw was in the design, not in any one product, so every vendor's software was vulnerable at once.

He called Paul Vixie, the longtime steward of the most widely used DNS software, and together they did something unusual. On March 31, 2008, engineers from the major DNS vendors gathered secretly at Microsoft's campus to agree on a fix.

The Secret Patch

The plan was for everyone to release patches on the same day, so that nobody could reverse-engineer one vendor's fix and attack the rest. On July 8, 2008, more than 80 vendors did exactly that, and Kaminsky asked the security community for a favor: give the world 30 days to patch before anyone speculated about the details.

It lasted 13 days. On July 21 a researcher guessed the mechanism on his blog, a security firm accidentally confirmed it, and two days later working exploit code was circulating.

By then enough of the internet had patched that the damage was contained. The "Kaminsky bug" became the textbook case for coordinated disclosure, and the reason the fix held is that so many people had already quietly installed it.

Roller Skates at Black Hat

In August 2008, Kaminsky walked onto the Black Hat stage in Las Vegas to explain the bug in full. He wore a suit, because his mother had asked him to, and roller skates, because he was Dan Kaminsky.

The room was the largest in the hotel and it was still standing room only. Afterward a man approached and introduced himself: the same military administrator who'd wanted him cut off at 11. He asked to be introduced to the meanest mother he'd ever met.

Conficker and the Keys to the Internet

Kaminsky's next big fix was for someone else's problem. In March 2009 the Conficker worm had infected millions of Windows machines, and nobody could tell from the outside which ones.

Poring over data with Tillmann Werner and Felix Leder of the Honeynet Project, he noticed that infected machines answered a particular network request differently from clean ones. Within days there was a free scanner, and the Honeynet Project's write-up gave every network defender a way to find the worm.

In June 2010 ICANN went further and made him part of the internet's own safety net. When the DNS root was signed for the first time, Kaminsky was named one of seven Recovery Key Share Holders, the trusted people from around the world whose key fragments could rebuild the master key if disaster struck.

White Ops and DanKam

Kaminsky kept turning down the obvious next step. Silicon Valley giants offered him lucrative jobs as chief information security officer, and he kept saying no.

Instead, in 2012 he co-founded White Ops, a company that fights the bots behind ad fraud, and became its chief scientist. The company renamed itself HUMAN in 2021 and still gives out a fellowship in his name.

His side projects were just as personal. He built DanKam, a phone app that recolors the world so colorblind users can tell its colors apart; after his grandmother lost her hearing, he worked on hearing-aid technology; and he built telemedicine tools with the National Institutes of Health for patients in underserved places.

Kaminsky's Death

Kaminsky died at home in San Francisco on April 23, 2021, of diabetic ketoacidosis, a complication of diabetes. He was 42.

The tributes came from everywhere, from the EFF, which called him a friend of freedom and the embodiment of the true hacker spirit, to the hackers who'd sat on the floor at his talks. Later that year the Internet Society inducted him into the Internet Hall of Fame.

Dan Kaminsky: The Man Who Patched the World

Kaminsky liked to say the internet was designed to move pictures of cats, and that some of us got to go out and fix it.

That's a modest way to describe a career that found the worst bug in the internet's foundations, got it patched before criminals could use it, and then handed the world the tools to measure its own infections.

Every time you type an address and land where you meant to, a little of that is him.