representation alignment
In machine learning, models often learn by imitating a pretrained teacher: they make their internal representations similar to the teacher's. This is useful because the teacher already knows good structure. But raw similarity can mislead: a teacher encodes both useful global knowledge and easy local details, so choosing what to imitate matters.
These papers show alignment works best when targets are chosen carefully. One selects a teacher layer whose features are hardest for the student to recover, weights harder tokens, and fades the loss. Another finds aligning to locally predictable content can hurt, and purifying that content helps. A third builds alignments from very few paired anchors plus many unpaired samples. A fourth uses semantic features to make video tokenizers more efficient. Together they turn alignment from simple copying into a principled design choice.
- Papers
- 5
- Released code
- 0
- First seen
- Oct 2026
- Latest
- Oct 2026
5 papers in the last two months, against 0 in the two before.
Start here
Three papers to read first, in the order they came out- 1Diffusion models benefit most from teacher features they still lack
RARE selects the teacher layer with the largest recoverability gap, improving FID and training speed.
Oct 2026 - 2Optimal transport aligns pretrained modalities with only hundreds of paired anchors
Aligning two frozen unimodal encoders requires only 100 to 500 paired anchors when anchor-relative coordinates, entropic optimal transport, and distillation are combined.
Oct 2026 - 3Purifying alignment targets improves diffusion generation, especially for reconstructive encoders
Aligning to the residual after removing locally predictable content (LAP) improves diffusion generation quality and turns MAE into an effective alignment target.
Oct 2026
How fast it is moving
Often studied with
diffusion transformerA neural network architecture combining transformer layers with diffusion processes for high-quality image and video generation.2 togetherflow matchingA generative training method that learns an ordinary differential equation mapping noise to data by matching velocity fields along probability paths.2 togetherdiffusion modelA type of generative model that learns to reverse a gradual noising process to create data.2 togetherThe papers
Most central to this idea first, not most recent.
- Top Universitycs.CV
Purifying alignment targets improves diffusion generation, especially for reconstructive encoders
Aligning to the residual after removing locally predictable content (LAP) improves diffusion generation quality and turns MAE into an effective alignment target.
Cornell University, University of Pennsylvania · Oct 2026
- Research Labcs.AI
Optimal transport aligns pretrained modalities with only hundreds of paired anchors
Aligning two frozen unimodal encoders requires only 100 to 500 paired anchors when anchor-relative coordinates, entropic optimal transport, and distillation are combined.
Sorbonne Université, Institut du Cerveau - Paris Brain Institute - ICM · Oct 2026
- Chinese Techcs.CV
Diffusion models benefit most from teacher features they still lack
RARE selects the teacher layer with the largest recoverability gap, improving FID and training speed.
Central South University, The Hong Kong University of Science and Technology · Oct 2026
- Chinese Techcs.CV
Image generators can train over 140 times faster with pixel feedback
SpeeDiff enables end-to-end training of latent diffusion models using a Tweedie loss that prevents latent collapse, achieving dramatically faster convergence.
ByteDance Seed, California Institute of Technology · Oct 2026
- AI Startupcs.CV
Semantic token prefixes let smaller autoregressive video models match larger ones
SemanTok, a flexible video tokenizer, achieves high semantic alignment and video fidelity, matching or beating models 3.4 times its size.
Stability AI, Karlsruhe Institut für Technologie · Oct 2026
Concepts are extracted from each paper and reused across the corpus, so this page grows on its own as the desk reads.