The authors froze pretrained generators and added trainable steering vectors, lists of numbers added to the outputs of model blocks.
Image generators recover most reward gains through fixed internal shifts
The authors reproduce most improvements from reinforcement learning by adding small, prompt-independent vectors to frozen image and video generators.
Big Tech
Tuna Han Salih Meral · Kaan Oktay · Alec Helbling · Atakan Kara · Duen Horng Chau · Pinar Yanardag
Virginia Tech · fal · Georgia Tech · IBM Research
Research Digest··2 min read

Before you read this
Three ideas this paper builds onactivation steeringActivation steering modifies a model's internal vector representations during inference to shift behavior, such as editing outputs, removing capabilities, or controlling trust.83 papers →reinforcement learningA machine learning method where an agent learns to make sequences of decisions by receiving rewards or penalties for its actions.301 papers →diffusion transformerA neural network architecture combining transformer layers with diffusion processes for high-quality image and video generation.76 papers →
Meral and colleagues show that much of reinforcement learning’s benefit for visual generators can be reproduced without changing their pretrained weights.
The Daily Edition
The front page in your inbox each weekday morning: the lead, the picks, and why each one earned its place.