Routing a query to a large language model usually means choosing which model to use based on how accurate it will probably be. But the cost of running a reasoning model varies hugely from one question to the next, the same model might write ten times as many tokens for a hard problem as for an easy one. Charging the average price for every query wastes money on short answers and may undercharge for long ones.
The researchers propose using a small, frozen LLM, called a scout, that only reads the prompt and predicts, before any model generates an answer, both how likely the candidate model is to succeed and how many tokens it will produce. The scout's predictions come from the hidden states of its early layers. This adds only a lightweight probe on top of an existing success-prediction router.
When they tested this with reasoning models on eight of nine benchmarks, the cost probe saved 9% to 33% of spending while keeping the same accuracy. The saving comes mostly from the fact that harder problems tend to produce longer outputs. The approach works with as few as twenty labeled examples and generalizes to new benchmarks without any labels.