The authors constructed ProtoArena, a black-box, multi-component benchmark with 80 cross-component vulnerability-exploitation environments spanning 11 protocol and application domains and 7 attack categories.
New benchmark tests LLM agents on exploiting cross-protocol vulnerabilities
ProtoArena includes 80 environments across 11 domains; Claude Opus 4.8 achieves 48.8% with guidance but only 12.5% under black-box conditions.
Chinese Tech
Fasheng Miao · Zhaoyu Hu · Xiang Li · Changqing An · Jiutian Zeng · Longtao Huang · +2 more
Tsinghua University · Alibaba Group · Nankai University · Quancheng Lab
Research Digest··3 min read
The authors introduce ProtoArena, a benchmark of 80 controlled environments designed to evaluate LLM agents' ability to exploit vulnerabilities arising from interactions among network protocol components.
Why this paper
From Alibaba Group and 3 others
In one line
LLM agents can exploit cross-component protocol vulnerabilities, with success rates from 48.8% (guided) to 12.5% (black-box).
What we could check
- ·No code link found
- ·No weights link found
- ·No dataset link found
- ·No compute details found
- ✓Limitations stated by the authors
- ✓Reports numbers on named benchmarks (2 benchmarks)
Observed from the paper text and links we have. Absence here means we did not find it, not that it does not exist.
§