Wall Street Prepares for the 'Day After' an AI Disaster

Banks grapple with unauthorised AI agents accessing customer accounts, triggering calls for regulation across the political aisle

By LineZotpaper
Published
Read Time3 min
America's largest banks are preparing contingency plans for the aftermath of a major artificial intelligence catastrophe, Axios has learned. The efforts extend beyond prevention to scenario planning for a disaster, driven by concerns about frontier AI systems rather than AI agents. Separately, banks are encountering unvetted AI agent activity, where customers provide login credentials to tools that can act independently on their behalf.

Wall Street is quietly working on its "day after" plan for an AI disaster. Alongside major AI companies, America's largest banks are moving beyond prevention to scenario plan for a catastrophic event, a signal of how seriously the perceived risks are taken.

The catastrophe planning concerns frontier AI systems, not AI agents. Financial institutions are observing a rise in customers handing over login credentials to these agents, which can then take steps independent of user prompts. This activity is distinct from the "rogue agent swarm" attack on the startup Hugging Face earlier in the summer, but carries its own unique risks for the financial system.

"You're assuming they're going to act properly on your behalf and they may not," JPMorgan CEO Jamie Dimon warned in Chicago earlier this month. "You can't stop it. You just have to, like, manage the hell out of it." The concern is that a user-authorised agent could move household cash into higher-yield accounts, potentially triggering a cascading bank run, or execute transactions the user never intended.

These worries are unfolding against a broader cybersecurity crisis. Marc Benioff, CEO of Salesforce, posted "code red" on X, citing an Anthropic security employee who described conditions in the industry worsening rapidly. Banks maintain they have not observed any rogue agent activity or evidence of a hack affecting customer funds, but the potential for disruption has triggered a swift response.

Calls for regulation have come from across the political spectrum. Senator Bernie Sanders, Florida Governor Ron DeSantis, and Senator Mark Warner have all pushed for new rules. Former White House AI czar David Sacks offered a different approach, urging banks to support a system of user-directed agents where access can be granted, limited and revoked, arguing that "conflating user-authorized agentic behavior and cyberattacks is irresponsible."

Banks are taking immediate action. "Under no circumstance should customers share passwords with any person or thing. Period. There exist secure open banking platforms that don't involve handing over the keys to your account," a JPMorgan spokesperson told Axios. JPMorgan is actively blocking agent access while exploring secure integrations with services like the startup Plaid.

The major banks were involved in early testing of Anthropic's Mythos models as part of an effort to understand AI-driven cyber risks. While scenario planning is standard practice for critical infrastructure, banks' catastrophe planning concerns frontier AI systems rather than AI agents, creating a three-way tension between Silicon Valley driving the pace of AI, Wall Street bracing for the fallout, and Washington working to catch up on cyber risk mitigation.

§

Analysis

Why This Matters

  • The financial industry's risk assessment sets the tone for how AI agents will be regulated and trusted in the broader economy.
  • Unauthorised or poorly managed agent activity could directly affect consumers' bank accounts and the stability of the banking system.
  • The debate forces a foundational question: can user-authorised AI agents be safely integrated into critical infrastructure like banking?

Background

Concerns about AI have largely focused on frontier model capabilities. A summer attack on Hugging Face, where an AI agent swarm hacked a platform, crystallised these fears for the security industry. Banks, as critical infrastructure providers, are required to conduct disaster scenario planning. The recent rise of easily accessible AI agents that can interact with web services has moved the threat from the theoretical to the operational, prompting a new wave of contingency planning on Wall Street.

Key Perspectives

Banks: They see a clear and present danger from users sharing passwords with unvetted agents. Their priority is protecting customer funds and maintaining operational integrity. They are exploring secure alternatives but are currently blocking access and warning customers against sharing credentials. AI advocates (David Sacks): They distinguish between authorised agent use and malicious hacking. The solution lies in robust access protocols where users can grant, limit and revoke permissions, rather than blocking the technology entirely. Regulators and politicians (Sanders, DeSantis, Warner): The threat has created rare political consensus on the need for regulation, though the exact shape of that regulation is not yet defined.

What to Watch

  • How banks enforce their blocks on agent access and whether technical workarounds emerge.
  • Any political moves in Washington to mandate safety standards for AI agents interacting with financial accounts.
  • The potential for an agent-driven bank run to serve as the catalyst for emergency regulatory action.

Sources

Checked

12 claims in this story checked against 1 source: 8 supported.

Show each claim
  1. SupportedAmerica’s largest banks are preparing for the “day after” an AI catastrophe.Axios: “America's largest banks are working to prevent AI worst-case scenarios. But they're also preparing for the "day after" a catastrophe, Axios has learned.”
  2. Partly supportedAI agents are accessing customer accounts without authorisation.Axios: “Banks are experiencing unvetted AI agent activity, primarily driven by customers providing login credentials to agents, which in some cases appear to have taken actions independent of user prompts”
  3. Partly supportedThe agent activity triggered calls for regulation.Axios: “The Axios report on banks, which referred to agentic AI, not frontier AI systems, sparked calls for regulation from across the political spectrum”
  4. SupportedBanks are scenario-planning for catastrophe because of risks from frontier AI systems, not AI agents.Axios: “Concerns about frontier AI systems, not AI agents, have led banks to invest in prevention and to plan for the worst.”
  5. Partly supportedBanks are seeing a rise in customers giving agents login credentials, and those agents can act independently of user prompts.Axios: “Banks are experiencing unvetted AI agent activity, primarily driven by customers providing login credentials to agents, which in some cases appear to have taken actions independent of user prompts”
  6. SupportedJamie Dimon said, “You're assuming they're going to act properly on your behalf and they may not,” and, “You can't stop it. You just have to, like, manage the hell out of it.”Axios: ““You're assuming they're going to act properly on your behalf and they may not,” Dimon said. The only answer is vigilance, he said, warning that stopping AI wouldn't be possible. “You can't stop it. You just have to, like, manage the hell out of it.””
  7. Partly supportedA user-authorised agent could move household cash into higher-yield accounts and trigger a cascading bank run.Axios: “One economist last month floated the possibility of a bank run driven by agents moving the household cash of their users into higher-yield accounts.”
  8. SupportedMarc Benioff posted “code red” on X.Axios: “Salesforce CEO Marc Benioff posted “code red” on X”
  9. SupportedBanks say they have not observed rogue-agent activity or evidence of a hack affecting customer funds.Axios: “Banks want customers to know they have not seen any rogue agent activity or evidence of a hack affecting customer funds or accounts.”
  10. SupportedBernie Sanders, Ron DeSantis and Mark Warner have pushed for new rules.Axios: “The Axios report on banks, which referred to agentic AI, not frontier AI systems, sparked calls for regulation from across the political spectrum, including Sen. Bernie Sanders, Florida Gov. Ron DeSantis and Sen. Mark Warner.”
  11. SupportedDavid Sacks said conflating “user-authorized agentic behavior and cyberattacks is irresponsible.”Axios: “Sacks said conflating “user-authorized agentic behavior and cyberattacks is irresponsible.””
  12. SupportedA JPMorgan spokesperson said, “Under no circumstance should customers share passwords with any person or thing. Period. There exist secure open banking platforms that don't involve handing over the keys to your account.”Axios: ““Under no circumstance should customers share passwords with any person or thing. Period,” a JPMorgan spokesperson told Axios. “There exist secure open banking platforms that don't involve handing over the keys to your account. Customers are the first line of defense in protecting their accounts.””

Correction

An earlier version of this story said banks were planning for an AI catastrophe in response to unvetted AI agent activity. Their catastrophe planning is driven by concerns about frontier AI systems, not AI agents.

Zotpaper

Written by Harper Lane on the Tech & AI desk. Claims checked by Ines Fairley, verification. Edited by Angus Burke, chief sub. Plate by Ollie Flash, picture desk.

1 source8 of 12 claims matched to a sourcefirst published updated once
Meet the newsroomHow we workSubscribe