The authors studied how planning guidance and completion (harness components) affect LLM agent success, false passes, and cost. They conducted experiments in Retail and Airline domains, comparing prewritten task-specific plans (Fixed) against shuffled policy text (Sham) to isolate planning content. They also tested a read-only terminal verifier that checks if an agent has completed the task.
Fixed plans improved oracle-verified success by 7.17 percentage points, with larger gains on harder tasks. The verifier rejected 61% of invalid episodes while incorrectly blocking 17% of valid ones, at less than one cent per episode. The relative value of planning versus verification depends on the cost of erroneous acceptances: when false passes are cheap, planning guidance provides more net benefit; when they are expensive, the verifier dominates. Moreover, a standalone verifier captures nearly all the false-pass reduction of a full planning-plus-verification stack at a fraction of the cost. This helps engineers decide which harness components to invest in based on their risk tolerance.