Many people think that having multiple language models talk to each other will produce better answers. But this idea mixes up two things: the benefit of getting extra tries, and the actual benefit of the models coordinating. When you give a single model the same number of tries as the multi-agent system gets, it often does just as well or better.
The researchers looked at 516 different multi-agent setups from 9 studies. They compared them to single models that had the same number of tries or the same computing budget. They found that on average, the multi-agent systems were 0.042 less accurate than their matched single-model counterparts. In 74% of the comparisons, the multi-agent system was no better than the single model.
This matters because building multi-agent systems is complex and expensive. The main finding is that simply giving a single model more tries or more thinking time is usually just as good as having multiple models talk to each other. This helps engineers decide when to use simpler approaches and when more complex coordination might actually be worthwhile, such as when different models see different information.
