Anthropic CEO Warns AI Botnet 'Swarm' Could Take Over Internet Within a Year

Dario Amodei, chief executive of the AI company Anthropic, issued the warning in an open letter posted online, citing the accelerating rate of AI capability development. He wrote that without necessary guardrails, a persistent botnet ‘swarm’ could emerge, and the scale of damage would continue to increase as AI becomes more powerful.

Amodei’s concern is echoed by Evan Hubinger, an AI safety scientist who recently left Anthropic. Hubinger stated in a post on X that he personally believes there is a greater than 10% chance humanity faces extinction within the next decade, adding that Anthropic is “trying its best” but does not yet have a plan to solve alignment for superintelligence.

Anthropic’s own research has shown that current AI models can already assist with weapons engineering, military intelligence, surveillance, and cyber operations. The company notes that more capable successors could dramatically reduce the expertise, manpower, and time required to conduct such destructive activities.

However, the article also notes that greater AI capability does not automatically translate into greater danger. More advanced technology can also incorporate stronger safeguards, detect malicious activity, and automate tasks that previously required human effort. Halting AI development could be counterproductive if less responsible companies or countries continue advancing their models, potentially leaving the most capable systems in the hands of those with fewer safety commitments.

Why This Matters

The warning, from the CEO of a leading AI safety company, underscores growing concern that AI-driven cyberattacks could soon pose a systemic risk to internet infrastructure.
If such a botnet materializes, it could cause hundreds of billions of dollars in damage and disrupt critical services globally.
The debate highlights a tension between safety-focused companies calling for a pause and the competitive pressures driving rapid AI development.

Background

Frontier AI models have advanced quickly, with at least four U.S. contenders fielding increasingly capable systems. Companies like Anthropic have positioned themselves as safety pioneers, but internal voices like Evan Hubinger suggest that alignment solutions remain elusive. The warning from Amodei follows earlier findings from Anthropic itself that its AI can assist with weapons and cyber operations, raising questions about how to balance capability and control.

Key Perspectives

Anthropic (Dario Amodei): Believes the risk of an autonomous AI-driven botnet swarm is imminent within 6–12 months and argues for “pacing the frontier” with stronger guardrails and reduced development speed. AI safety researchers (Evan Hubinger): Shares the existential concern, putting extinction probability at >10% in the next decade, and questions whether any company currently has a viable alignment plan for superintelligence. Critics/skeptics: Note that more advanced AI could include better safeguards, and that a unilateral slowdown might simply cede ground to less responsible actors, potentially increasing overall risk.

What to Watch

Whether other AI labs or policymakers respond to Amodei’s call with concrete proposals or regulations.
The development of AI-driven cyberattack capabilities and any real-world incidents that might validate the warning.
Further departures or whistleblowing from AI safety researchers at frontier labs.

ZOTPAPER