The authors designed EvalRx as a closed-loop system that starts from a fixed labeled evaluation set.
An agentic framework that diagnoses and verifiably repairs foundation model failures
EvalRx uses statistically grounded hypothesis testing and tiered interventions to improve performance across language, vision-language, and audio-language models.
Top University
Jiaqi Liu · Ruinan Jin · Yitong Tang · Beidi Zhao · Gexin Huang · Qiong Zhang · +1 more
The University of British Columbia · Vector Institute · Mohamed bin Zayed University of Artificial Intelligence (MBZUAI)
Research Digest··3 min read
The authors introduce EvalRx, an open-source agentic framework that discovers systematic failure modes in foundation models, formulates falsifiable hypotheses, and verifies them before applying targeted repairs.
§