The authors developed Kaggle Game Arena, infrastructure for scheduling model competitions, integrating game environments, recording outcomes and presenting results.
Competitive games provide a dynamic benchmark for language model strategy
Kaggle Game Arena evaluates models through head-to-head chess, poker and Werewolf competitions with objectively scored outcomes.
Independent
Bovard Doerschuk-Tiberi · Yao Yan · Justin Chiu · Hann Wang · Timothy Chung · Martyna Plomecka · +56 more
Research Digest··2 min read
Thread:Process Agent Benchmarks
The authors built an extensible evaluation platform in which language models compete directly in structured games rather than answer a fixed test set.
Why this paper
Independent
In one line
LLMs can be objectively evaluated through competitive games like Chess, Poker, and Werewolf to avoid benchmark saturation.
What we could check
- ·No code link found
- ·No weights link found
- ·No dataset link found
- ·No compute details found
- ·No stated limitations found
- ·No benchmark numbers found
Observed from the paper text and links we have. Absence here means we did not find it, not that it does not exist.
§