The authors introduce Interaction-as-a-Judge, in which a computer-use agent operates an application and records a multimodal trace containing actions and visual responses.
Interactive software judges improve when they test applications directly
Computer-use agents that operated generated applications predicted human preferences, verified requirements and guided repairs better than code-only evaluators.
Independent
Wayne Chi · I-Hung Hsu · Li Chen · Wei-Lin Chiang · Ion Stoica · Anastasios Nikolas Angelopoulos · +3 more
Research Digest··2 min read
Chi and colleagues tested whether an automated judge should use software as a person would, rather than assess only its source code or static screenshots.
Why this paper
Independent
In one line
Interaction-as-a-Judge using computer-use agents improves evaluation and generation of interactive applications over code-only methods.
What we could check
- ·No code link found
- ·No weights link found
- ·No dataset link found
- ·No compute details found
- ✓Limitations stated by the authors
- ✓Reports numbers on named benchmarks (2 benchmarks)
Observed from the paper text and links we have. Absence here means we did not find it, not that it does not exist.
§