The authors train activation monitors on LLM internal states, varying the computational context of the prompts (rewording, changing formatting) while keeping the target model and monitoring architecture fixed.
Activation monitors gain only modest advantage from original model computations
Predicting model behavior from activations works nearly as well with altered transcripts, but native access yields small consistent improvements.
Top University
James Oldfield · Shawn Im · Jay J Culligan · Philip Torr · Sharon Li · Adel Bibi
University of Oxford · University of Wisconsin–Madison · Toyota Motor Europe
Research Digest··2 min read
The authors investigate whether activation monitors benefit from access to a model's original computations.
§