Adaptive inference systems choose which model to use for each input to balance accuracy and cost. Training models and the router separately is inefficient because each model trains on all data even though the router may send it only a subset. The authors propose new hinge losses for jointly training classifiers and the router in two common setups: single-stage (one choice per input) and cascade (sequential decisions). They prove these losses are Bayes-consistent, meaning minimizing them recovers the best possible cost-sensitive accuracy. They also introduce a method using unlabeled data to fix routing errors when labeled data is scarce. The hinge losses provide theoretical guarantees and work well in practice, giving a principled foundation for training efficient adaptive systems.
New hinge losses ensure optimal routing in adaptive AI classification
The authors prove that minimizing these losses recovers the Bayes-optimal balance between accuracy and inference cost for both flat and cascade routers.
Big Tech
Shirley Xiaoqi Liu · Florence Regol · Antonios Valkanas · Patrick Rebeschini · Mark Coates
University of Oxford · McGill University · Amazon
Research Digest··3 min read
In one line
Cost-sensitive hinge losses enable Bayes-consistent joint training of classifiers and routers.
Before you read this
Three ideas this paper builds onmodel routingDirecting each query or subtask to the most suitable model to optimize cost, accuracy, or quality.74 papers →confidence calibrationConfidence calibration aligns a model's predicted confidence with its actual accuracy across predictions.67 papers →privileged informationInformation available only during training that improves performance but is not accessible at inference.64 papers →
Read it as
Liu et al. propose cost-sensitive hinge losses for jointly training classifiers and routers in adaptive inference systems.
The authors formalized adaptive inference in classification with two routing models: single-stage (flat) routers that select one classifier per input based on base features, and cascade routers that sequentially decide whether to exit or defer to a more costly classifier.
How it works
before and afterThe Daily Edition
The front page in your inbox each weekday morning: the lead, the picks, and why each one earned its place.