6 papers this week in Tools & MCP12 active threadsbusiest: Agent Security & Attacks

Tools & MCP research

Evaluation & benchmarks

Language models can write small networks from scratch but cannot safely edit large ones

Marasini et al. introduce WeightBench, a benchmark for systematically evaluating how well large language models can read, fix, and write neural network weights without gradient descent. Their results show that frontier agents can create simple control policies from scratch, but cannot yet patch large language models, compress them without data, or tell impossible edits from hard ones.

5 days ago
Evaluation & benchmarks

Web agents handle audio and video, but best systems still fail on cross-site tasks

The authors introduce WebArena-Pro, a benchmark of 300 human-validated web agent tasks across 20 self-hosted applications in six domain categories. It extends evaluation to include audio and video as core observations, and runs each task in container isolation for reproducible parallel evaluation. The best performing agents achieve 62.3% success, revealing that cross-site tasks remain significantly harder than single-site ones.

6 days ago
Evaluation & benchmarks

Simulated business histories yield verifiable benchmarks that separate AI agent performance

The authors present SynthCorp, a platform that simulates how business events create and modify corporate data over time, producing relational databases, document stores, and data catalogs alongside evaluation tasks with verified gold programs. In tests across five domains, the benchmarks consistently separate different data contexts and model tiers, and an auditing method using the simulation log automatically verifies gold programs without manual annotation, even on composite tasks that current state-of-the-art agents fail.

9 days ago