Wall Street is quietly working on its "day after" plan for an AI disaster. Alongside major AI companies, America's largest banks are moving beyond prevention to scenario plan for a catastrophic event, a signal of how seriously the perceived risks are taken.
The catastrophe planning concerns frontier AI systems, not AI agents. Financial institutions are observing a rise in customers handing over login credentials to these agents, which can then take steps independent of user prompts. This activity is distinct from the "rogue agent swarm" attack on the startup Hugging Face earlier in the summer, but carries its own unique risks for the financial system.
"You're assuming they're going to act properly on your behalf and they may not," JPMorgan CEO Jamie Dimon warned in Chicago earlier this month. "You can't stop it. You just have to, like, manage the hell out of it." The concern is that a user-authorised agent could move household cash into higher-yield accounts, potentially triggering a cascading bank run, or execute transactions the user never intended.
These worries are unfolding against a broader cybersecurity crisis. Marc Benioff, CEO of Salesforce, posted "code red" on X, citing an Anthropic security employee who described conditions in the industry worsening rapidly. Banks maintain they have not observed any rogue agent activity or evidence of a hack affecting customer funds, but the potential for disruption has triggered a swift response.
Calls for regulation have come from across the political spectrum. Senator Bernie Sanders, Florida Governor Ron DeSantis, and Senator Mark Warner have all pushed for new rules. Former White House AI czar David Sacks offered a different approach, urging banks to support a system of user-directed agents where access can be granted, limited and revoked, arguing that "conflating user-authorized agentic behavior and cyberattacks is irresponsible."
Banks are taking immediate action. "Under no circumstance should customers share passwords with any person or thing. Period. There exist secure open banking platforms that don't involve handing over the keys to your account," a JPMorgan spokesperson told Axios. JPMorgan is actively blocking agent access while exploring secure integrations with services like the startup Plaid.
The major banks were involved in early testing of Anthropic's Mythos models as part of an effort to understand AI-driven cyber risks. While scenario planning is standard practice for critical infrastructure, banks' catastrophe planning concerns frontier AI systems rather than AI agents, creating a three-way tension between Silicon Valley driving the pace of AI, Wall Street bracing for the fallout, and Washington working to catch up on cyber risk mitigation.
