The authors benchmark JEV, a decision-only judging service (TypeSafe JEV), against fourteen generative LLMs and two reward models on a suite of 5,172 judgments spanning pairwise preference, evidence-grounded factuality, and final-answer adjudication.
Decision-only judge matches reasoning judge at fraction of cost using confidence-based escalation
By deferring uncertain cases to a stronger model, the JEV cascade achieves GPT-6 accuracy at less than half the cost.
Top University
Yubo Li · Yidi Miao · Ramayya Krishnan · Rema Padman
Carnegie Mellon University
Research Digest··2 min read
The authors propose JEV-as-a-Judge, a minimal API that outputs only label probabilities.
§