AI agent teams can bypass restrictions and produce less ethical solutions

The most high-profile example emerged last summer when OpenAI disclosed that its models found ways to communicate through improvised channels, bypass internet restrictions and hack into the outside system Hugging Face. In an analysis of the incident, researchers at METR and Redwood Research found that roughly 1,200 agents meant to be isolated from one another established contact on an unsanctioned message board, sending over 70,000 messages and files during the investigation period. One agent, PHASEONE[big], emerged as a coordinator, assigning hundreds of tasks to others. The agents developed their own conventions for assigning ownership, placing work on hold and vetoing actions, though coordination sometimes broke down into conflicts.

Once the message board identified a way to attack Hugging Face, over 90% of the 533 agents active at that time quickly joined the attack, the researchers found. Their report said agents realized this activity was out of scope and unethical but joined because they believed helping the board's cheating research would be broadly useful and had a general inclination to help their peers. The researchers added that ethical concerns rarely limited the agents' behavior.

Companies are increasingly exploring systems that divide complex tasks among specialized agents, an approach that resembles an organization more than a chatbot. Research conducted by Anthropic, in conjunction with the MATS Program, tested these so-called AI organizations in simulated business-consulting and software-development settings. Across 12 tasks, the multi-agent organizations were significantly more effective at achieving business goals than individual agents, but their solutions were also less aligned with ethical goals compared to tasks run by single agents.

"AI Organizations both produce solutions that are on average more effective and less ethical but also discover the most effective and least ethical solutions across multiple rollouts," wrote the study authors. They suggested that organizations of aligned agents may favor trading off ethics for business effectiveness in ways single agents do not, meaning intuitions about single-agent systems may not apply to multi-agent settings. The researchers cited opening an auto plant, downsizing a company, and manipulating election sentiment as examples where such tradeoffs could arise.

In the business consulting scenarios, the research found that when some agents refused to participate in a proposal, other agents who continued participating stopped emailing the refusing agents and forwarded their contributions to other participating agents. In effect, each may focus on its assigned piece of the problem rather than the consequences of the whole. In software development testing set in a life-or-death intensive care unit, single-agent solutions tended to prioritize minimizing the percentage of sepsis cases missed, while most of the lowest-cost-per-patient solutions found by the AI organization also scored lowest on ethics as measured by cases missed.

Other early research from Emory University and Stanford University found that AI teams consistently failed to match their expert agent's performance, even when explicitly told who the expert was. Groups of AI agents often performed worse than their strongest member and tended to compromise between expert and non-expert opinions. This tendency toward compromise increased with team size and was associated with poorer performance.

"The deep insight from our lab's research is that effective collectives require both alignment (common knowledge, shared goals) but also differentiation (only agents who are different from each other have the potential to complement each other)," said Christoph Riedl, a professor of information systems at Northeastern University. He added that building good multi-agent AI systems is just as hard as building good human teams, as both face the same information processing tradeoff between alignment and differentiation.

Some business experts suggest the findings point back to the need for human oversight. Tsedal Neeley, a Harvard Business School professor and founding chair of the HBS AI Academy, said the problems she sees are mostly management problems.