Anthropic warns AI could pose existential risk in $US100b listing push

Anthropic's filing says sufficiently advanced AI systems could pose catastrophic or existential risks to humanity, particularly if misused, replicated, weaponised or allowed to operate with a high degree of autonomy. In the same document, the company argues that AI is "one of the most consequential technologies in human history", one that could accelerate scientific innovation, greatly increase economic growth rates, bring cures for major diseases within reach and solve problems that have resisted generations of human effort.

The warning comes amid a global AI construction boom that has drawn comparisons with the canal boom of the 1830s, the railroad boom of the late 19th century, the rollout of electricity, the federal highway system and the dotcom rush. This year alone an estimated US$1 trillion will be spent on data centres and the computer servers inside them. In Australia, PwC analysts estimate data centre spending could reach $157 billion by the turn of the decade and $342 billion by 2036. If the boom accelerates, spending could reach $428 billion or more than 10 per cent of GDP. Federal Treasury forecast non-mining investment to grow just 1 per cent through 2025-26; it grew 14 per cent, thanks to the AI boom.

Columbia University economist Stijn Van Nieuwerburgh estimated last month that AI infrastructure spending would equal 3.63 per cent of US GDP between 2025 and 2032, larger than the 2.24 per cent share of GDP for the late-19th-century railroad boom. Nvidia, which produces hardware vital to AI, is currently the world's most valuable company at US$5.2 trillion, and 13 of the 19 companies worth more than US$1 trillion are tied to AI. A successful listing would make Anthropic the 20th company worth more than US$1 trillion.

Economists are also questioning how the boom is financed. Bank for International Settlements chief economist Phurichai Rungcharoenkitkul has examined what AI companies call “circular financing”, in which a large tech company or chipmaker invests or lends to an AI start-up that must then buy chips or cloud services from the same company. By some accounts, up to 55 per cent of investment in AI firms comes from other AI businesses. He found boom-bust patterns recur across history, from US canal mania in the 1830s to the dotcom boom of the 1990s, and concluded that the AI boom “creates fragility that undermines itself”: the more capacity the sector builds, the higher the productivity bar it must clear to sustain the boom.

Fitch Ratings put a number on the risk: a 35 per cent drop in share prices over six months would push the US into recession quickly, with the economy 3.2 per cent smaller within a year. Market expectations are already under scrutiny. Alphabet, which owns Google and YouTube, saw its shares fall 10 per cent this week despite revenue from its AI cloud business jumping 108 per cent, as investors weighed its plan to spend up to US$200 billion on infrastructure next year. Reserve Bank assistant governor Brad Jones noted the “exuberance” around tech companies, saying earnings expectations for the US IT sector beyond the next three years had, until recently, been running at about 45 per cent a year in perpetuity.

The boom is already showing up in prices. Spending by IT companies on equipment has soared 220 per cent over the past year, and computer equipment prices have risen 6.3 per cent after falling 91 per cent between 2000 and 2025. Australian inflation data this week showed prices for games and hobbies up 15.5 per cent over the year to September, the fastest since the bureau began tracking the sector in 1998, because gaming companies are competing with cashed-up AI firms for computer chips. The construction is also drawing local opposition, with US communities facing higher utility bills as new data centres become their noisy neighbours.

ANZ chief economist Richard Yetsenga said AI and the construction boom underpinning it would ultimately deliver long-term economic and productivity improvements that could dwarf previous technological advances, and that the infrastructure being built would last for years. Every new form of technology has prompted new policy imperatives that need to be resolved, and AI seems the same, he said. Supporters such as Elon Musk argue AI could bring extraordinary economic growth, while critics warn it could lead to mass unemployment as the technology replaces workers. Australian Signals Directorate director-general Abigail Bradshaw said AI technologies had changed the cybersecurity landscape, enabling malicious actors to identify vulnerabilities and attack faster and at greater scale. The agency believes AI will help it prevent the avalanche of attacks.

Anthropic says that if AI is directed at humanity's hardest problems and its risks are managed, “the potential is extraordinary”.