joint-embedding predictive architecture
Joint-embedding predictive architecture, or JEPA, is a method where a system learns to represent the world by predicting future states in a learned abstract space, rather than generating exact images or videos. This lets the system focus on task-relevant patterns and ignore irrelevant visual details, making it efficient for planning and decision-making in robotics and autonomous driving.
The papers show that JEPA can be made more useful for control by aligning its predictions with the actual dynamics and tasks. They address failures like losing information about unstable motions, generating impossible sub-goals, or mismatching a robot's body. Solutions include reconstructing action sequences from latent states, retrieving sub-goals from real observations, sparsifying representations, and adapting at test time using all deployment data including failures. Across tasks, matching the predictive teacher to the control problem consistently improves success, robustness, and inference speed more than simply scaling up data or compute.
- Papers
- 28
- Released code
- 2
- First seen
- Sept 2026
- Latest
- Nov 2026
28 papers in the last two months, against 0 in the two before.
Start here
Three papers to read first, in the order they came out- 1AI models can align language and vision using 42-fold fewer samples
Joint-Embedding Predictive Architectures with good views provably learn the same representation across modalities up to a rotation.
Oct 2026 - 2Robot policy outperforms larger generative models with a quarter of the parameters
La-JEPA trains robot policies from scratch in a pretrained JEPA feature space and matches or beats much larger generative-backbone policies with fewer parameters and faster inference.
Nov 2026 - 3Stochastic latent variable improves masked image prediction without hurting representations
Making the predictor's hidden variable stochastic in Joint-Embedding Predictive Architectures improves representation quality and adds uncertainty estimates and diverse predictions.
Nov 2026
How fast it is moving
Often studied with
latent world modelA learned internal, compressed representation of environment dynamics, enabling future-aware planning, generation, and reasoning directly in latent space.15 togetherworld model for agentsA predictive environment model that agents use to simulate outcomes, enabling planning, reasoning, and long-horizon task execution.9 togetherself-supervised learningA paradigm where models learn representations from unlabeled data by predicting hidden or transformed parts of the input.7 togetherWho is working on it
The papers
Most central to this idea first, not most recent.
- Academiccs.AI
AI models can align language and vision using 42-fold fewer samples
Joint-Embedding Predictive Architectures with good views provably learn the same representation across modalities up to a rotation.
Brown University, AMI Labs · Oct 2026
- Chinese Techcs.RO
Robot policy outperforms larger generative models with a quarter of the parameters
La-JEPA trains robot policies from scratch in a pretrained JEPA feature space and matches or beats much larger generative-backbone policies with fewer parameters and faster inference.
Huawei Technologies Ltd., University of Toronto · Nov 2026
- Top Universitycs.LG
Stochastic latent variable improves masked image prediction without hurting representations
Making the predictor's hidden variable stochastic in Joint-Embedding Predictive Architectures improves representation quality and adds uncertainty estimates and diverse predictions.
Imperial College London, Information Technologies Institute, CERTH · Nov 2026
- Chinese Techcs.AI
Mobile AI agents run up to four times faster, sacrificing accuracy
A JEPA-based GUI agent picks actions by scoring latent representations of candidate elements, running several times faster than a 7B vision-language model without decoding tokens.
Shanghai Jiao Tong University, Nanyang Technological University · Oct 2026
- Top Universitycs.AI
Latent world models for robotics scale predictably with compute
Latent robotic world model quality and downstream planning success follow a second-order power law in training compute across 12 embodiments.
FAIR at Meta, Chandar Research Lab · Oct 2026
- Research Labcs.LG
Dependency sparsity can isolate world latents without reconstructing observations
Under Structural Diversity, DSReg recovers individual world latents up to signed permutation without reconstruction, decoders, labels, or non-Gaussian assumptions.
University of Illinois Urbana-Champaign, Cold Spring Harbor Laboratory · Oct 2026
- Top Universitycs.LG
Inverse dynamics helps visual world models retain unstable control modes
Adding an inverse dynamics loss to JEPA world models preserves unstable control modes that standard prediction objectives collapse.
Columbia University, New York University · Oct 2026
- Academiccs.LG
SCOPE recovers complete PDE fields from sparse observations via mask-conditioned latent prediction.
SCOPE recovers complete PDE fields from sparse observations using full-target latent prediction and shared physical reconstruction.
Stony Brook University, University of California, Davis · Oct 2026
- Academiccs.RO
One action-conditioned JEPA improves vision-language-action policies across all lifecycle stages
Juno uses a single action-conditioned JEPA to improve VLA success rates across pretraining, policy learning, and deployment.
University of Science and Technology of China, Eastern Institute of Technology, Ningbo · Oct 2026
- Big Techcs.AI
Retrieving real states improves long-horizon planning over generated latent sub-goals
Retrieving sub-goals from recorded episodes outperforms generating latent sub-goals for long-horizon planning.
Samsung AI, University of Edinburgh · Sept 2026
- Academiccs.ROcode
Compressed world models make zero-shot driving plans far faster
AD-E2E-JEPA compresses latent driving representations to accelerate world-model planning 100-fold while preserving planning quality and improving downstream imitation learning.
New York University, AMI Labs · Oct 2026
- Chinese Techcs.CV
Latent trajectory guidance improves video generation reasoning
VR-JEPA uses latent trajectory prediction to guide video generation for improved visual reasoning.
Sun Yat-sen University, Shenzhen Loop Area Institute · Oct 2026
- Big Techcs.SD
AI audio models better recognize individual sounds and whole soundscapes
AudioFM achieves state-of-the-art audio representation learning by combining recording-level contrast with patch-wise masked latent discrimination.
Google Research, Google DeepMind · Oct 2026
- Big Techcs.CV
One video model jointly tracks 3D hands and camera motion
V-JEPA Hand jointly recovers bilateral hand poses, camera motion, and focal lengths from egocentric clips in one forward pass, outperforming baselines on three benchmarks.
Tsinghua University, Microsoft Research · Oct 2026
- Big Techcs.RO
Action-aware embeddings retrieve better robot experiences for future actions
Robota-Embed jointly encodes vision, language, and action for robot experience retrieval, outperforming action-free embeddings.
NVIDIA, MIT · Oct 2026
- Big Techcs.AI
Explicit predictive states boost physical plausibility in video world models
Physical Registry adds explicit predictive states to video DiTs, improving physical plausibility with minimal parameters.
Yonsei University, Microsoft Research Asia · Oct 2026
- Top Universitycs.LG
Spectral alignment lets users specify what world models preserve
Spectral alignment of embedding geometry to a target similarity kernel improves world model planning performance.
Stanford University, ETH Zürich · Oct 2026
- Top Universitycs.LG
Hierarchical latent models improve long-horizon visual planning with less compute
Hierarchical world models with separate latent spaces at each level improve long-horizon planning success and efficiency over flat models.
NYU, Advanced Machine Intelligence · Oct 2026
- Academiccs.LGcode
Symplectic latent dynamics prevent rollout contraction in visual world models
A symplectic Hamiltonian flow as the JEPA predictor preserves latent phase-space volume and substantially improves long-horizon velocity prediction on pendulum environments.
Courant Institute, New York University, Brown University · Oct 2026
- Top Universitycs.CV
A single flow matching Transformer generates images and text jointly in pretrained encoder spaces
A single flow matching transformer over frozen encoder spaces generates both images and text jointly without fixed modality order.
New York University, Harvard University · Oct 2026
- Big Techcs.CV
Video models reason better when trained to anticipate future states
ProAR makes autoregressive video reasoning more goal-directed by predicting outcomes and aligning current representations with future transitions, improving accuracy and training efficiency.
The Hong Kong Polytechnic University, University of California, Davis · Oct 2026
- Big Techcs.LG
Flexible action chunks boost world model planning for distant goals
FlexiWorld achieves 89.29% mean success via variable-length action chunks and mixed-span goal supervision.
University of Toronto, Zhejiang University · Oct 2026
- Top University
Action-discriminative world models improve counterfactual planning for robot control
World models for planning should preserve action-dependent differences for counterfactual selection, not just factual prediction accuracy.
Nanjing University · Sept 2026
- Big Techcs.CV
Driving simulators follow planned paths more faithfully with overhead views
BirdDreams jointly generates perspective and bird's-eye-view videos to create reactive driving simulations from a single image.
NVIDIA, KAIST · Nov 2026
- Chinese Techcs.CV
Inverse dynamics initializes better action sequences for visual planners
DIAL improves latent planning by training an inverse dynamics model to initialize action search.
Huazhong University of Science and Technology, Tencent ARC Lab · Nov 2026
- Top Universitycs.AI
AI agents can learn from imagined outcomes without generating images
Dwell reframes a vision-language model as a recurrent latent transition operator to learn from experience.
HKUST, National University of Singapore · Oct 2026
- Research Labcs.LG
World models learn true dynamics with fewer parameters
World2vec learns faithful, compact world models from observations and actions without rewards, matching reward-trained control and identifying dynamical systems.
Helmholtz Munich, Munich Center for Machine Learning (MCML) · Oct 2026
- Chinese Techcs.RO
Systematic study reveals what matters for world modeling in robot VLAs
2D semantic representations and decoupled world model integration improve VLA robustness on long-horizon and dynamic tasks.
Chinese Academy of Sciences, University of Chinese Academy of Sciences · Oct 2026
Concepts are extracted from each paper and reused across the corpus, so this page grows on its own as the desk reads.