The authors define the Behavioral Necessity Rate (BNR), the proportion of initially correct cases in which deleting targeted evidence prevents a model from recovering the answer.
Correct Answers Often Fail to Prove Genuine Multi-Hop Reasoning
Behavioral interventions reveal whether models actually depend on every evidence step that a benchmark claims is necessary.
Chinese Tech
Jiawen Tao · Xiaokun Yuan · Yaoming Li · Chenxu Liu · Mengzhou Wu · Tong Yang · +1 more
Hunyuan Team Tencent · Peking University
Research Digest··3 min read
Tao et al.
Why this paper
From Hunyuan Team Tencent and Peking University
In one line
Correct answers often bypass intended multi-hop reasoning chains, while evidence-deletion auditing and REALHOP produce evaluations that depend on every annotated hop.
What we could check
- ·No code link found
- ·No weights link found
- ·No dataset link found
- ·No compute details found
- ·No stated limitations found
- ✓Reports numbers on named benchmarks (5 benchmarks)
Observed from the paper text and links we have. Absence here means we did not find it, not that it does not exist.
§