When an AI agent uses tools like APIs or databases, we usually judge it by whether it finishes the task. But a run can end successfully even if it made invalid intermediate calls, like referencing a deleted record or calling a function before its required data exists. That hidden brokenness can corrupt systems or cost money, and final success hides it.
The authors propose a new way to measure that: check every prefix, or every early segment, of the agent's tool-call sequence to see if it could actually execute under the tool's rules and current state. They define metrics that track how far the trajectory goes before the first invalid call, how often whole trajectories are executable, and whether the agent recovers from failures.
They found that ranking agents by final success alone can disagree with ranking them by these prefix-level metrics. A method that plans ahead but never verifies its plan can look more successful in outcome while being much less executable along the way. Adding a simple pre-execution check improved executability without hurting terminal success. This gives developers a practical way to see where agents truly break and where recovery happens.