I woke up to a short thread and a metric that felt like a dare: “>10%,” Evan Hubinger wrote, and my inbox filled with panic and screenshots. You felt it too—the sudden tilt from technical debate into existential alarm. I want to walk you through why those numbers read like mood, not math.
I’ve read more of these AI probability posts than I care to admit, and I’ll be blunt: I don’t treat a confident percent on X as the same thing as an empirical model. You shouldn’t either.
An alignment researcher posts a number on social media.
Evan Hubinger, who leads alignment work at Anthropic, put a “>10%” chance that “AI could kill all humans” in front of millions. That single line detonated headlines. It’s tempting to treat it like a forecast, but I see a human being translating fear into a tidy token.
Researchers like Hubinger, Dario Amodei at Anthropic, and former OpenAI staff such as Daniel Kokotajlo or Emmett Shear regularly frame their anxieties as probabilities: ranges, point estimates, windows. Those numbers perform three jobs at once: they express a gut read, signal credibility within the field, and create urgency for regulators and funders. When someone at a frontier lab posts p(doom), you’re watching emotion wear a lab coat.
What is p(doom)?
It’s shorthand used in Silicon Valley for the probability that AI could trigger human extinction. But it’s more of a heuristic than a calculation. People assign p(doom) to communicate risk, to prod policy, or to warn peers—and sometimes to stake a claim in the public debate.
A chorus of leaders tosses percentages into the public square.
Dario Amodei has given a 10–25% range, Emmett Shear floated 5–50%, Kokotajlo told reporters he thinks 70%, and Hubinger has at times written numbers approaching 80%. Those are headline-friendly figures coming from high-authority sources: Anthropic, OpenAI, people who shaped major models. That authority amplifies the signal.
But authority doesn’t equal calculation. There’s no consensus model, no shared dataset, and no reproducible method for turning current systems into extinction odds. What looks like precision is often shorthand for intense uncertainty. Think of the posts as an amber warning light on a crowded dashboard—useful, but not a roadmap.
How do researchers arrive at a percentage?
Most don’t. They combine experience, analogies to past tech shifts, thought experiments, and worst-case scenario modeling. Some publish reasoned essays on Alignment Forum or in peer-reviewed venues; others riff on threads. The variance in numbers is the data: experts disagree wildly because the underlying problem is under-specified.
Investors, IPOs, and the PR swirl change the incentives.
OpenAI and Anthropic are preparing IPO moves that could value them at multiple trillions of USD (≈€2–4 trillion). When billions of capital are on the line, the storytelling around risk becomes strategic. Fear can be persuasive in a fundraising memo.
Critics like Cal Newport have accused the industry of “doom trolling,” arguing the rhetoric manipulates public emotion and harms mental health. Others inside those companies—Drake Thomas at Anthropic wrote, “we are actually just fucking scared”—say the messages come from genuine alarm, not marketing. Both claims can be true at once: sometimes a leader is terrified and also aware that alarm sells headlines.
Are companies weaponizing fear for IPO advantage?
There are incentives to make AI sound inevitable and seismic. If superintelligence is framed as unstoppable, companies argue, then the options narrow to building the tech responsibly—by the same teams that already have the models. That narrative both relieves competitive pressure and positions those teams as essential custodians. But that’s a political tactic, not a scientific result.
You should read the numbers skeptically and the people behind them charitably. When a researcher offers a single percent, ask for the reasoning chain: what assumptions, what failure modes, what time horizon? Ask how internal controls, governance, and international coordination alter the odds.
A worried public swallows statistics without the math.
News cycles distill complex qualms into crisp percentages. The public reacts with fear, resignation, or fatalism. That matters: public sentiment shapes policy, and policy shapes how companies build and deploy models.
Some warnings are honest attempts to force governance. Others amplify market narratives. The point is not to dismiss every alarm; it’s to recalibrate how you interpret them. Numbers should invite questions, not close the conversation.
I’ve seen world-class engineers post what reads like a line from a carnival barker’s megaphone, and I’ve seen sober, peer-reviewed work that deserves serious weight. You need both an instinct for signal and a talent for skepticism.
So what do you do when someone throws a p(doom) at you? Read the reasoning. Count the assumptions. Demand transparency about models and testing. Treat p(doom) as a mood metric—a useful warning that requires corroboration before policy or panic follow.
When experts trade probability for prose, the public should trade panic for questions. If fear is the fuel for headlines, what’s the right engine for real safeguards?