The authors evaluated activation steering on seven language models using When2Call and MetaTool, benchmarks for deciding whether and how a model should use a tool.
Steering tool decisions helps, but correction does not guarantee repair
SAKIKO tracks where activation-steered decisions end up, revealing gains that aggregate scores can hide or misclassify.
Research Lab
Jiayi Li · Ruizhe Li
University of the Chinese Academy of Sciences · University of Birmingham
Research Digest··2 min read
Thread:Agent Auditability
Li and Li introduce SAKIKO, a framework for auditing internal activation interventions that alter whether language models call tools, seek clarification, answer directly or decline.
Why this paper
From University of the Chinese Academy of Sciences and University of Birmingham
In one line
Internal activation steering can increase correct tool calls but corrupts over half of baseline-correct decisions; repair requires destination-resolved auditing.
What we could check
- ·No code link found
- ·No weights link found
- ·No dataset link found
- ·No compute details found
- ✓Limitations stated by the authors
- ·No benchmark numbers found
Observed from the paper text and links we have. Absence here means we did not find it, not that it does not exist.
§