The paper tests whether LLM agents filling the same role in different teams are interchangeable, as production systems assume. Eight teams were formed independently from one base model on identical tasks, each agent keeping a private notebook across ten episodes. After forming, role-matched agents were swapped between teams, and performance was measured on held-out tasks against a placebo that mimicked the disruption without changing the agent.
Task scores changed little after a swap, but communication per unit of progress rose by 16 to 63 percent. In the Hanabi task, a swapped agent was actually more costly than an inexperienced one, consistent with interference from conventions learned with its former partner. In Collab-Overcooked, when the agenda-setting agent was replaced, most extra communication came from the agent that stayed.
Three ablations (varying base model, decoding temperature, and formation length) showed that the swap penalty moved together with how far independently formed teams drifted apart: greedy decoding lowered both, while longer team histories raised both. The finding is interesting because it reveals that LLM agent teams develop partner-specific coordination patterns that are disrupted by replacement, challenging the common assumption that role-based agents are freely interchangeable.