Stochastic latent variable improves masked image prediction without hurting representations

S-JEPA uses a learned prior over plausible completions to generate diverse, uncertainty-aware predictions, outperforming deterministic baselines like I-JEPA and LeJEPA.

Top University

Nikos Giakoumoglou · Paschalis Giakoumoglou · Tania Stathaki

Imperial College London · Information Technologies Institute, CERTH

Research Digest·9 November 2026·2 min read

Illustration: Stochastic latent variable improves masked image prediction without hurting representations

In one line

Making the predictor's hidden variable stochastic in Joint-Embedding Predictive Architectures improves representation quality and adds uncertainty estimates and diverse predictions.

Before you read this

Three ideas this paper builds on

joint-embedding predictive architectureAn encoder predicts the latent representation of future data points from current and past observations, bypassing pixel-level reconstruction.26 papers →self-supervised learningA paradigm where models learn representations from unlabeled data by predicting hidden or transformed parts of the input.55 papers →confidence calibrationConfidence calibration aligns a model's predicted confidence with its actual accuracy across predictions.86 papers →

Read it as

When a model learns to understand images by guessing missing parts in a high-level way, it often has to pick just one guess for what is missing, even when many answers would be valid. This wastes information and limits the model's ability to know when it is unsure.

The researchers created S-JEPA, which instead lets the model learn a range of plausible guesses for the missing part. It does this by adding a source of randomness that is tuned during training, so the model learns to represent many possible completions without being told the right answer.

The key result is that this added randomness makes the model slightly better at core tasks while giving it useful abilities the deterministic version lacks: it can produce uncertainty maps that correlate with its errors, and generate varied outputs for the same input, all at no cost to overall performance.

How it works

before and after

RSS

The Daily Edition

The front page in your inbox each weekday morning: the lead, the picks, and why each one earned its place.

Concepts

joint-embedding predictive architecture stochastic latent variable modeling conditional variational autoencoder self-supervised learning predictive uncertainty estimation confidence calibration