Fri 13 Nov
LiveGet it free
ZOTPAPER

News without the noise

← All concepts

latent world model

A latent world model is a compressed internal simulation. Instead of predicting raw pixels, it learns to predict hidden states in a compact representation. "Latent" means the hidden state, and the model uses past information and possible actions to imagine future outcomes. This lets agents plan toward goals, avoid costly trial and error, and generate coherent behavior without seeing every visual detail.

The papers show a common shift from reactive to anticipatory decision-making. They predict future states at multiple time scales, sometimes hierarchically, and often with frozen or reusable components. Planning inside a latent space generalizes across tasks and platforms, reduces compute, and improves reliability. Some work finds that predictive accuracy improves predictably with training effort, making latent imagination a useful proxy for real-world performance. Others use learned prior trajectories or foresight signals to replace expensive online search, suggesting that latent world models support not just prediction but efficient, reusable planning.

Papers
51
Released code
4
First seen
June 2026
Latest
Nov 2026

44 papers in the last two months, against 5 in the two before.

Start here

Three papers to read first, in the order they came out
  1. 1AI agents can learn visual tasks through longer imagined futures

    MAC-v2 uses action chunking and reconstruction-free latent modeling to enable long-horizon value expansion from pixels.

    Nov 2026
  2. 2Robot policy outperforms larger generative models with a quarter of the parameters

    La-JEPA trains robot policies from scratch in a pretrained JEPA feature space and matches or beats much larger generative-backbone policies with fewer parameters and faster inference.

    Nov 2026
  3. 3Inverse dynamics initializes better action sequences for visual planners

    DIAL improves latent planning by training an inverse dynamics model to initialize action search.

    Nov 2026

How fast it is moving

JunJulAugSeptOctNov so far

Often studied with

joint-embedding predictive architectureAn encoder predicts the latent representation of future data points from current and past observations, bypassing pixel-level reconstruction.15 togetherworld model for agentsA predictive environment model that agents use to simulate outcomes, enabling planning, reasoning, and long-horizon task execution.10 togethervision-language-action modelA robot policy model that maps visual observations and language instructions to actions, trained on demonstrations and often used for manipulation tasks.7 together

Who is working on it

NVIDIA 4University of Chinese Academy of Sciences 3University of Toronto 2UC Berkeley 2Huazhong University of Science and Technology 2Cold Spring Harbor Laboratory 2

The papers

Most central to this idea first, not most recent.

Concepts are extracted from each paper and reused across the corpus, so this page grows on its own as the desk reads.