The authors propose that an agent harness can delegate routine multiple-choice decisions, such as which tool to call, to a small decision model while reserving harder cases for the LLM.
Small decision models can safely handle routine agent decisions with little loss of success
The authors show that fitting confidence thresholds on an agent's own trajectories lets a 4B model cut cost by 81.1-89.0% and latency by up to 2.5x without major success loss.
Big Tech
Kunlun Zhu · Tao Feng · Xu Han · Heng Wang · Ling Yang · Xiangru Tang · +2 more
University of Illinois Urbana-Champaign · Princeton University · Google
Research Digest··2 min read
Thread:Agent Harness Optimization
The authors build AgentDecisionBench, JevAgentHarness, and AgentDecision-4B to test when a fast decision model can take over routine choices inside an LLM agent.
Why this paper
From Google and 2 others · Part of Agent Harness Optimization, now 143 papers
In one line
Replacing most LLM calls with a decision model cuts agent cost and latency with minimal success loss.
What we could check
- ·No code link found
- ·No weights link found
- ·No dataset link found
- ·No compute details found
- ·No stated limitations found
- ·No benchmark numbers found
Observed from the paper text and links we have. Absence here means we did not find it, not that it does not exist.
§