Benchmark errors inflate and deflate AI policy-reasoning scores

Audit of nine policy-based decision benchmarks finds that correcting defects in references and evaluators can change acceptance rates by over 80 percentage points without altering model responses

Big Tech

Yanru Xiao · Yifei Cai

Amazon Web Services · Iowa State University

Research Digest·6 November 2026·2 min read

Illustration: Benchmark errors inflate and deflate AI policy-reasoning scores

In one line

Policy-based decision benchmarks have defects that misjudge rule-following responses, impacting scores and comparisons.

Before you read this

Three ideas this paper builds on

benchmarkA standard set of tasks or data used to evaluate and compare the performance of AI systems.322 papers →audit workflowA systematic process for verifying data integrity, model behavior, and evaluation validity through checks for errors, confounds, and hidden artifacts.100 papers →evidence blindnessHaving access to relevant evidence does not prevent an LLM agent from making incorrect or unsafe decisions.27 papers →

Read it as

test if systems apply written rules, but mistakes in the benchmark can mark correct answers as wrong. We audited nine such benchmarks by tracing sources, running programs, and checking references. We found errors in supplied answers and scoring.

Correcting the scorer on a legal benchmark raised acceptance by over 80 percentage points for each model. On a tax benchmark, fixing a rule changed 27 of 300 references and narrowed a model gap. This shows that benchmark flaws affect system evaluations.

How it works

before and after

RSS

The Daily Edition

The front page in your inbox each weekday morning: the lead, the picks, and why each one earned its place.

Concepts

benchmark audit workflow evidence blindness failure attribution