self-supervised learning
Self-supervised learning trains models on unlabeled data by turning the data into its own teacher. An image model may predict a hidden patch or align two views of the same image; an audio model may separate overlapping speakers or recover masked sound. The model learns useful representations without human labels.
Across papers, the same recipe works in different domains. In vision, aligning global views creates semantic structure, and local crops or masking help specific tasks. In video, separating appearance from motion gives efficient temporal encoders. In speech, forcing language discrimination or a source-separation objective shrinks gaps. Audio models combine global and local objectives. Even reasoning models can learn to predict their confidence, and efficiency improves as a side effect.
- Papers
- 60
- Released code
- 1
- First seen
- Sept 2026
- Latest
- Nov 2026
60 papers in the last two months, against 0 in the two before.
Start here
Three papers to read first, in the order they came out- 1Self-improving agent learns video-to-action control from environment interaction alone
Mirror learns video-to-action control from environment interaction alone, following visual plans without rewards or expert action labels and outperforming prior learning-from-observation methods.
Oct 2026 - 2AI audio models better recognize individual sounds and whole soundscapes
AudioFM achieves state-of-the-art audio representation learning by combining recording-level contrast with patch-wise masked latent discrimination.
Oct 2026 - 3AI models can improve image understanding and generation from their outputs
Deciding per sample whether text or image is the more reliable half of a self-generated multimodal trajectory lets a unified model improve both understanding and generation without outside supervision.
Nov 2026
How fast it is moving
Often studied with
contrastive alignmentA method that learns embeddings by minimizing distance between positive pairs and maximizing distance between negative pairs.8 togethervision-language modelA model that handles tasks combining visual and textual information, such as editing images from conversation or reasoning across images and text.7 togetherjoint-embedding predictive architectureAn encoder predicts the latent representation of future data points from current and past observations, bypassing pixel-level reconstruction.7 togetherWho is working on it
The papers
Most central to this idea first, not most recent.
- Chinese Techcs.CV
AI models can improve image understanding and generation from their outputs
Deciding per sample whether text or image is the more reliable half of a self-generated multimodal trajectory lets a unified model improve both understanding and generation without outside supervision.
Xi’an Jiaotong University, Qwen DianJin Team, Alibaba Cloud Computing · Nov 2026
- Academiccs.AI
Self-improving agent learns video-to-action control from environment interaction alone
Mirror learns video-to-action control from environment interaction alone, following visual plans without rewards or expert action labels and outperforming prior learning-from-observation methods.
Massachusetts Institute of Technology · Oct 2026
- Big Techcs.SD
AI audio models better recognize individual sounds and whole soundscapes
AudioFM achieves state-of-the-art audio representation learning by combining recording-level contrast with patch-wise masked latent discrimination.
Google Research, Google DeepMind · Oct 2026
- Top Universitycs.CL
LongZero trains LLMs on self-written long-context tasks, no external data
LongZero improves LLM long-context reasoning via self-generated training tasks without any external data.
Tsinghua University, Tsinghua-Bosch Joint ML Center · Oct 2026
- Big Techcs.CV
Video pretraining decouples motion and appearance paths to boost motion tasks
TT-VidT decouples appearance and motion in video pretraining, leading on motion-heavy benchmark fine-tuning with fewer FLOPs.
National Tsing Hua University, Comfy Org Research · Oct 2026
- Research Labcs.CV
Two same-image global views drive semantic structure in DINO models
DINO-style semantic structure comes from aligning two geometrically distinct global crops of the same image; masking refines it, while local crops mainly improve classification.
CISPA Helmholtz Center for Information Security, University of Freiburg · Oct 2026
- Big Techcs.CV
Patch-level self-supervision scales to seven billion parameters with simpler objectives
JEM uses a principled patch-level objective to train stable visual encoders up to 7B parameters while outperforming DINOv2 on dense prediction tasks.
Meta FAIR · Oct 2026
- Independenteess.AS
Mask-free separation pretraining improves representations of overlapping speech
SepRQ achieves state-of-the-art multi-speaker representation learning using mask-free pseudo-source-separation with frozen codebooks.
Oct 2026
- Top Universitycs.AI
Teaching Models Confidence Can Shorten Reasoning Without Explicit Stop Training
Self-supervised confidence fine-tuning reduces generated tokens by up to 25% at matched accuracy without directly optimizing for shorter reasoning.
University of Maryland, AI Foundations, Capital One · Sept 2026
- Top Universitycs.CV
Self-supervised tri-modal model perceives and forecasts any road obstacle without labels
TriO is a tri-modal unsupervised world model that predicts 4D occupancy, obstacle segmentation, flow, and LiDAR, achieving state-of-the-art on multiple 3D/4D tasks and zero-shot road obstacle segmentation.
Waabi, University of Toronto · Oct 2026
- Independentcs.CV
Depth AI fails a simple zoom test meant for camera-distinguishing models
Metric depth models mistake digital zoom for camera movement because they rely on object size.
Nov 2026
- Big Techcs.LG
Language models can beat answer-key training by solving easier problems
Label-free self-distillation outperforms voting by using easier problems the model solves consistently as teacher context.
Amazon, Worcester Polytechnic Institute · Nov 2026
- Industrycs.AI
Debate traces yield useful training data when confidence replaces voting
AbCD converts multi-agent debate traces into preference data by clustering answers and ranking by confidence, improving reasoning without gold labels.
Upwork Inc. · Nov 2026
- Top Universitycs.LG
Stochastic latent variable improves masked image prediction without hurting representations
Making the predictor's hidden variable stochastic in Joint-Embedding Predictive Architectures improves representation quality and adds uncertainty estimates and diverse predictions.
Imperial College London, Information Technologies Institute, CERTH · Nov 2026
- Chinese Techcs.LG
A frozen AI model can imitate an unseen dataset from one example
A pretrained generator can create realistic new time series for an unseen domain from a single reference sequence, without retraining, by reusing learned shape and shape-transition vocabularies.
State Key Laboratory of AI Safety, Institute of Computing Technology, Chinese Academy of Sciences, University of Chinese Academy of Sciences · Nov 2026
- Top Universitycs.RO
Unlabelled navigation videos can teach robots how scenes change as they move
OpenNWM learns universal latent actions from action-free video to generalize navigation world models across environments.
CASIA, University of Chinese Academy of Sciences · Nov 2026
- Top Universitycs.CV
Feature-trained image tokens improve generation quality at lower sampling cost
Training discrete image tokens with feature reconstruction instead of pixel reconstruction improves generative quality and efficiency.
Mila - Québec AI Institute, Université de Montréal · Oct 2026
- Chinese Techcs.CV
AI video generators reduce flickering and preserve detail with few steps
Adding a predictive representation loss to distribution matching distillation improves few-step video generation spatial fidelity and temporal coherence.
Shanghai Jiao Tong University, Hunyuan (Tencent) · Oct 2026
- Big Techcs.CV
Progressive depth growth improves vision transformers with less compute
Progressive depth growth in self-supervised vision transformers improves downstream performance while cutting pretraining FLOPs by 13.9-24.8%.
KTH Royal Institute of Technology, Helmholtz AI · Oct 2026
- Research Labcs.CV
Discrete part concepts from frozen self-supervised backbones match base accuracy without labels
DisParQ learns discrete, spatially grounded part concepts and attributes from frozen self-supervised vision models without labels or language, matching teacher accuracy.
Jagiellonian University, Max Planck Institute for Informatics · Oct 2026
- Big Techcs.CV
Generative chroma keying from unaligned background references preserves foreground performance
FreeCHROMA generates composite videos from unaligned background references while preserving foreground identity and motion.
University of Maryland, College Park, Amazon · Oct 2026
- Big Techcs.LG
Pairing ECGs with multiple clinical report types yields a more universal ECG foundation model
Training on multiple clinical note types improves ECG foundation model performance beyond using only ECG reports.
Massachusetts Institute of Technology, Apple, Inc · Oct 2026
- Big Techcs.AI
Training on self-explanations improves coding agents without reinforcement learning
Fine-tuning an agent on its own self-generated retrospections improves task performance without external rewards or RL.
RPI, UC San Diego · Sept 2026
- Top Universitycs.LG
Semantic alignment improves edge classification across unseen dynamic graph domains
Spatio-Temporal Semantic Alignment outperforms all existing methods on edge classification for dynamic text-attributed graphs.
Imperial College London · Sept 2026
- Big Techcs.CL
Language discrimination training reduces the multilingual gap in speech models
Strengthening language discrimination during pretraining closes the multilingual gap in speech models.
Apple · Oct 2026
- Research Labcs.AI
Temporal distance improves self-supervised goal-directed reinforcement learning
ChronoSRL gives critic embeddings temporal geometry, training distances to match goal-reaching times for faster and more reliable learning.
Technical University of Darmstadt, Robotics Institute Germany (RIG) · Sept 2026
- Big Techcs.CL
Language models improve using self-generated questions and answers, without human examples
Large language models self-improve using only self-generated questions and privileged reasoning, without human-curated data.
Google, Ulm University · Oct 2026
- Big Techcs.CV
Unified visual encoder delivers state, language, and reconstruction for world models
ALPS is a visual encoder for world models that jointly learns state information, language alignment, and pixel reconstruction from still images and text.
University of California, Santa Cruz, University of California, Santa Barbara · Oct 2026
- Top Universitycs.CL
Reward models for vision-language AI can be trained without human preference labels
At 32B, a linear probe on a frozen VLM trained on caption perturbations with no human labels beats GPT-4o on VL-RewardBench.
Massachusetts Institute of Technology, Cornell University · Oct 2026
- Big Techcs.CV
Raw video alone, with no captions, can mid-train language models for better vision.
Mid-training a pretrained language model on raw video through next-visual-token prediction improves image and video understanding while preserving text performance, without captions.
Meta AI, Massachusetts Institute of Technology · Oct 2026
- Independentcs.SE
Existing scientific software can generate verified training tasks for terminal agents
Existing scientific software workflows provide self-supervised training data for terminal agents via software-in-the-loop reconstruction.
Oct 2026
- Top Universitycs.RO
DexCast learns real-world dexterous control from human videos without teleoperation
DexCast learns dexterous robotic hand control from monocular human videos without teleoperation, achieving 87.5% success on real-world tasks.
Tsinghua University, TsingTec · Oct 2026
- Chinese Techcs.CV
Future chunk prediction improves physical plausibility in autoregressive video generation
Adding Future Chunk Prediction as an auxiliary training objective improves physical plausibility in autoregressive video generation without changing inference.
Nanjing University, ARC Lab, Tencent IEG · Oct 2026
- Academiccs.LG
SCOPE recovers complete PDE fields from sparse observations via mask-conditioned latent prediction.
SCOPE recovers complete PDE fields from sparse observations using full-target latent prediction and shared physical reconstruction.
Stony Brook University, University of California, Davis · Oct 2026
- Top Universitycs.CV
Training vision tokens for semantic consistency boosts multimodal AI on visual tasks
Multimodal large language models benefit from semantically consistent vision tokens; adding semantic guidance to training improves vision-centric benchmarks while preserving text-centric performance.
KU Leuven · Nov 2026
- Independentcs.SD
AI music dataset contains duplicate versions in over one-fifth of files
At least 38,134 files in the Lakh MIDI dataset are duplicates, and filtered lists are provided.
Nov 2026
- Big Techcs.CL
UltraBERT balances masking strategies to improve classification and retrieval
UltraBERT is the first encoder to achieve state-of-the-art on both classification and retrieval tasks.
Korea University, AIGEN Sciences · Oct 2026
- Big Techcs.CV
One video model jointly tracks 3D hands and camera motion
V-JEPA Hand jointly recovers bilateral hand poses, camera motion, and focal lengths from egocentric clips in one forward pass, outperforming baselines on three benchmarks.
Tsinghua University, Microsoft Research · Oct 2026
- Academiccs.LG
Scaling law for physiological waveforms favors more training data over larger models
Larger physiological waveform models and more pretraining hours reduce the need for labeled clinical data.
University of North Carolina at Chapel Hill · Oct 2026
- Chinese Techcs.LG
Test-time calibration learning improves LLM accuracy and confidence without labels
An LLM's confidence can be calibrated at test time without labels using self-supervised rewards from multiple generated answers.
Hong Kong Baptist University, Huawei Noah's Ark Lab · Oct 2026
- Big Techcs.SD
Audio training teaches one model to edit scenes across modalities
Training on synthetic audio edits and masked audiovisual reconstruction enables zero-shot, instruction-driven editing of movie scenes across image, audio, and video modalities.
Adobe Research, Carnegie Mellon University · Oct 2026
- Big Techcs.CV
MRI foundation model aligns anatomy and abnormalities across 3D volumes
A 3D MRI vision-language model that aligns image-text representations at the anatomical level and separates healthy from abnormal anatomy outperforms existing MRI foundation models.
GE Healthcare, Amazon · Oct 2026
- Chinese Techcs.CV
Small feature adapters remove visual domains while preserving scene content
A small adapter on frozen DINO features moves a 13-14% residue to translate domains and outperforms pixel-fed generators with fewer parameters.
Huawei, Graz University of Technology · Oct 2026
- Big Techcs.LG
Random exploration yields planning without reward or policy improvement training
Random exploration data enables long-range planning by learning temporal log-density ratios at multiple horizons without policy improvement training.
UCLA, University of Minnesota · Oct 2026
- Academiccs.ROcode
Compressed world models make zero-shot driving plans far faster
AD-E2E-JEPA compresses latent driving representations to accelerate world-model planning 100-fold while preserving planning quality and improving downstream imitation learning.
New York University, AMI Labs · Oct 2026
- Big Techcs.RO
Robots can learn manipulation from human video via four paradigms
Ego2Robot learning classifies four ways egocentric human video helps train robot manipulation policies.
The Hong Kong Polytechnic University, NVIDIA · Nov 2026
- Chinese Techcs.LG
Fitting regression models to tokens improves compression in two settings
Test-time ridge regression compresses token sequences, outperforming pooling and projection on retrieval and visual merging.
The Chinese University of Hong Kong, Independent Researcher · Oct 2026
- Industrycs.CV
AI models can answer image questions and refine pictures they generate
A unified discrete diffusion LLM can work for both understanding and generating images, using self-refinement and cross-paradigm distillation to match specialized models.
Oct 2026
- Academiccs.AI
Image models can outperform specialized models at reading brain recordings
A frozen vision model that has never seen EEG outperforms specialized EEG models when signals are rendered as images.
Rutgers University–New Brunswick · Nov 2026
- Big Techcs.CV
Selecting ViT tokens cuts labels needed for semantic segmentation
Pixel-level active learning for semantic segmentation works best when candidate selection happens in ViT token space, not interpolated pixel space.
University of Waterloo, Apple · Oct 2026
- Top Universitycs.LG
Spectral alignment lets users specify what world models preserve
Spectral alignment of embedding geometry to a target similarity kernel improves world model planning performance.
Stanford University, ETH Zürich · Oct 2026
- Academiccs.CV
Aligning partial point clouds to frozen global 3D encoders without labels
PAGER aligns partial 3D point cloud features to a frozen global semantic space using paired geometry, recovering performance lost to coordinate-frame mismatch.
Technical University of Denmark, Pioneer Center for Artificial Intelligence · Oct 2026
- Academiccs.RO
Stereo module lifts 2D vision-language-action models to 3D for robot manipulation
ExStereo augments pre-trained 2D Vision-Language-Action models with 3D perception using stereo vision, improving robotic manipulation success rates.
University of Southern California · Oct 2026
- Chinese Techcs.RO
Robots can use human videos before learning their own controls
Robot policies can learn reusable, three-dimensional motion representations from videos without robot command labels, then adapt them to different robot controls.
Hong Kong Embodied AI Lab, CUHK · Oct 2026
- Independentcs.AI
Clustering keys makes 10-million-token attention 22x faster with similar accuracy
Nov 2026
- Chinese Techcs.AI
Mobile AI agents run up to four times faster, sacrificing accuracy
A JEPA-based GUI agent picks actions by scoring latent representations of candidate elements, running several times faster than a 7B vision-language model without decoding tokens.
Shanghai Jiao Tong University, Nanyang Technological University · Oct 2026
- Independenteess.AS
Choosing speech encoder layers on evaluation data inflates depression scores
Choosing speech encoder layers on evaluation folds inflates depression-detection AUC, while nested selection removes the bias and favors a compact affect-prosody representation.
Oct 2026
- Research Labcs.CV
One point cloud registration model generalizes across indoor, outdoor, and object domains
Register Anything Model (RAM) unifies point cloud registration across diverse domains with state-of-the-art results.
Defense Innovation Institute, Academy of Military Sciences, China, College of Computer Science and Technology, National University of Defense Technology, China · Oct 2026
- Academiccs.AI
Vision-language alignment fails to improve text-only spatial reasoning
VQA-style vision-language alignment does not improve text-only reasoning on visualization tasks; self-distillation of visual embeddings into text does.
New York University · Oct 2026
- Chinese Techcs.SD
Audio-language model suite adds speaker-aware transcription and understanding
Qwen-Audio-3.1-ASR improves multilingual speech recognition and adds speaker-labeled, timestamped transcription and broader audio understanding through variant-specific capabilities.
Alibaba Group · Oct 2026
Concepts are extracted from each paper and reused across the corpus, so this page grows on its own as the desk reads.