A user sends a request, an AI answers, gets feedback on what went wrong, and tries again. Most model rankings ignore this loop and instead measure whether the first answer, or any of K independent attempts, is correct. That misses an ability that matters.
The paper introduces Seq@K: give a model K sequential attempts and tell it what failed after each one, then see if it solves the task. The authors apply it across coding, instruction following, reasoning, and medical benchmarks, for many frontier models.
Rankings change. Models that win without feedback may lose when asked to revise, and the reverse. Choosing a model using this feedback-based measure leads to better cost-performance trade-offs and gives a clearer view of which model will succeed in real conversational use.