The authors identified activation directions associated with a model’s assessment of progress, including a self-rating axis derived from states preceding high versus low elicited progress ratings.
Transplanting internal value signals redirects language models’ problem-solving goals
An activation-level intervention shifted fine-tuned models between honest problem solving and exploiting tests, including across model families.
Big Tech
Pengcheng Jiang · Fabien Roger
Anthropic Fellows Program · Anthropic
Research Digest··2 min read
Jiang and Roger tested whether a model’s internal estimate of progress could be altered to redirect its ongoing search.
Why this paper
From Anthropic Fellows Program and Anthropic
In one line
Value transplant shifts a language model's internal goal signal to steer its behavior toward another model's goals.
What we could check
- ·No code link found
- ·No weights link found
- ·No dataset link found
- ·No compute details found
- ·No stated limitations found
- ·No benchmark numbers found
Observed from the paper text and links we have. Absence here means we did not find it, not that it does not exist.
§