The paper addresses the problem of routing queries to the best specialist model in a heterogeneous AI system, where estimating each specialist's value is costly. Cheap estimates are fast but noisy, while accurate estimates are expensive. The authors formalize this as a Pandora's Box problem, where the router can pay to inspect a specialist's true value. They propose Pandora's Router, a centralized policy that uses closed-form value-of-information expressions to decide when to pay for a better estimate. Experiments show Pandora's Router matches the routing quality of exhaustive estimation while querying the expensive estimator far less often. The paper also introduces Pandora's Bidder for decentralized settings where specialists independently decide whether to invest in self-assessment. This work connects to classical search theory, offering a principled way to manage the cost-accuracy tradeoff in value estimation.
Cost-aware model routing matches exhaustive estimation with fewer expensive checks
A Pandora’s Box formulation lets routing systems decide when a more accurate assessment of a model’s expected performance is worth its computational cost.
Big Tech
Adam Fisch · Shubhendu Trivedi · Fantine Huot · William W. Cohen · Michael Kaisers · Mirella Lapata · +2 more
Google DeepMind
Research Digest··2 min read
Thread:Agent Harness Optimization
In one line
Routing queries to AI models can optimally trade off estimation cost and accuracy using Pandora's Box search policies.
Before you read this
An idea this paper builds onRead it as
Fisch et al. model AI routing as a search problem in which inspecting each candidate model’s likely value is itself costly.
The authors formulate routing among heterogeneous AI specialists as an instance of the classical Pandora’s Box problem: a router has uncertain estimates of each specialist’s expected return and must decide whether paying for a better estimate is worthwhile before assigning a query.
The Daily Edition
The front page in your inbox each weekday morning: the lead, the picks, and why each one earned its place.