The evaluation compares otherwise identical multi-stage LLM pipelines: in the strict version, each stage sees only the previous stage’s output; in the full version, every stage also sees the original problem. Across 21 open-weight models on GSM-Hard and MATH-500, many comparisons showed a statistically significant decomposition tax, reaching 40.5 accuracy points.
The loss comes from interfaces discarding problem information, not merely from token count. Instructions that preserve relationships between quantities reduce the tax, while re-grounding works best at the stage immediately after the lossy interface. This matters because stronger and newer models still suffer large losses, so model upgrades alone do not make decomposed pipelines reliable.