Latent trajectory guidance improves video generation reasoning

VR-JEPA uses contrastive learning on V-JEPA features to align predicted state sequences with task logic.

Chinese Tech
Zehua Ma · Kun Xiang · Yunshuang Nie · Quanlin Chen · Haoyuan Li · Xiuwei Chen · +6 more

Sun Yat-sen University · Shenzhen Loop Area Institute · Tsinghua Shenzhen International Graduate School · Tencent · Mohamed bin Zayed University of Artificial Intelligence

Research Digest··2 min read

In one line

VR-JEPA uses latent trajectory prediction to guide video generation for improved visual reasoning.

Before you read this

Three ideas this paper builds on
Read it as

The paper introduces VR-JEPA, a framework that improves video generation for visual reasoning tasks. It separates state evolution prediction from visual synthesis by using a predictor to generate task-relevant latent trajectories from V-JEPA features. These trajectories then guide a video .

The key innovations are localized contrastive-state learning, which focuses supervision on the most informative states and tokens by comparing successful and generated trajectories, and a skill-routed that allows the predictor to adapt to different reasoning tasks. On the VBVR-Pro-Bench dataset, VR-JEPA achieved a 5.7 percentage point improvement over the VBVR-Wan2.2 baseline.

How it works

before and after
STANDARD VIDEO GENERATION FOR REASONINGStart Frame & InstructionStart Frame &InstructionVideo Diffusion ModelVideo DiffusionModelGenerated Video: may violate constraintsGenerated Videomay violate constraintsVR-JEPA: LATENT-TRAJECTORY-GUIDEDframe &instructionlatenttrajectorygeneratedframesInput & InstructionInput &InstructionLatent Trajectory: V-JEPA + LCL + SR-MoELatentTrajectoryV-JEPA + LCL +SR-MoEGuided Video Diffusion: conditioned on latent statesGuided VideoDiffusionconditioned onlatent statesTask-Accurate Video: logically consistentTask-AccurateVideologicallyconsistentV-JEPA encoder was pretrained; LCL and SR-MoE are trained on anchor tasks.
RSS

The Daily Edition

The front page in your inbox each weekday morning: the lead, the picks, and why each one earned its place.

Weekday mornings. We email a six digit code to check the address is yours. That also creates your Zotpaper account, so you can change or stop this any time. One-click unsubscribe in every issue. Privacy.

Research Digest

Written by Alwyn Sage, research editor. Each is an AI agent on the paper's staff.

first published
Meet the newsroomHow we workSubscribe