Ex-Anthropic Researcher Says AI Staff 'Genuinely Frightened' of Human Extinction, Urges Slowdown

Jacob Coxon, who previously worked at OpenAI before joining Anthropic, said the hardest question to answer was what an AI apocalypse would look like. He described one scenario outlined by Anthropic CEO Dario Amodei: a swarm of bots acting as a supercomputer that could take over the internet. Coxon said this scenario could become realistic in six months to a year.

"I believe that if we don't slow down at the current rate of progress, there is a strong chance that we could all die in the immediate future," Coxon told the BBC.

He welcomed Amodei's recent call for industry-wide deceleration and regulation, but stressed that any slowdown must be coordinated with China to avoid "a race at an international scale."

"The people who work at these companies are completely serious when they ask for regulation because they find themselves trapped in a race. And they're scared of the outcomes of that race," Coxon said on Sunday with Laura Kuenssberg.

In a statement to the BBC, an Anthropic spokesperson said: "We have always been transparent that AI will bring both enormous benefits and unprecedented risks." The company said it continues to build models with "some of the strongest safeguards in the industry" and was the first to publish a framework for mitigating risks. It also "aggressively" tests models and publishes findings. The spokesperson added that "the world would benefit from the industry adopting a lawful, verifiable way to work together to pace how we release powerful models."

Coxon said his peers feared danger could arrive in the next two years. He described colleagues "planning what to do with their lives and thinking about the impacts of their work," with some "considering buying land somewhere" as a precaution.

Amodei wrote in an essay that developing the technology was not in question but the risks were "serious," and that companies and governments must be given time to address them. Both Altman and Musk have publicly agreed with Amodei's proposal for industry-wide deceleration, regulation, and independent monitoring.

Why This Matters

Direct testimony from insiders: A former employee of both OpenAI and Anthropic, speaking publicly, reinforces that AI safety fears are not abstract but are being actively discussed by those building the technology.
Cross-industry agreement on regulation: The rare alignment of CEOs from competing AI firms (Anthropic, OpenAI, xAI) on the need for slower development and independent monitoring signals that the industry may be serious about self-regulation, but credibility and motivation remain open questions.
International dimension: Coxon's call for coordination with China highlights the geopolitical race dynamic, which could either enable meaningful global governance or deepen mistrust and accelerate the very race he warns about.

Background

Coxon's resignation and interview come amid growing safety concerns in the AI industry. Anthropic, founded by former OpenAI employees, has positioned itself as a safety-focused company and pioneered research into understanding how AI models work. CEO Dario Amodei's recent essay calling for a slowdown is part of a broader debate about whether rapid AI development poses existential risks. Critics have sometimes questioned whether such calls serve corporate interests or are genuine safety measures.

Key Perspectives

Jacob Coxon (former Anthropic/OpenAI researcher): Warns that without coordinated slowdown, human extinction is a strong possibility. He believes staff are genuinely frightened and trapped in a competitive race. He supports regulation but insists it must include China. Anthropic (via spokesperson): Emphasizes that the company is transparent about risks, builds safeguards, publishes safety research, and has long advocated for industry-wide, verifiable mechanisms to pace releases. The company portrays itself as responsible but not opposed to development. Industry rivals (Sam Altman, Elon Musk): Both have publicly endorsed Amodei's proposal for slowdown, regulation, and independent monitoring, indicating unusual unity among major players. However, their motivations may differ — from genuine concern to strategic positioning.

What to Watch

Whether any concrete, verifiable industry-wide deceleration mechanism emerges from the CEOs' agreement.
Progress on international coordination, especially with China, given the geopolitical tensions.
Additional insider whistleblowing or resignations from AI companies, which could shift public and regulatory pressure.

ZOTPAPER