SWE-DIVE converts existing software-engineering benchmark tasks, such as those in SWE-bench, into incremental subtasks.
Simulated users test coding agents as requirements unfold across conversations
SWE-DIVE turns existing software tasks into adaptive conversations, with an 11-task preliminary evaluation reporting distinct interaction patterns and higher regression rates under unconstrained or changing requirements.
Big Tech
Lorenz Brehme · Benjamin Steenhoek · Ruth Breu
University of Innsbruck · Microsoft
Research Digest··2 min read
Brehme and colleagues introduce SWE-DIVE, a framework for evaluating coding agents through conversations rather than a single complete task description.
§