The paper presents a method to generate high-resolution stereoscopic 360° video from monocular panoramic video without requiring stereo training data. It extends existing video by first warping the left-eye view to a right-eye view using estimated spherical depth, then using the diffusion model to fill in missing regions and fix inconsistencies. A new metric called PEGS (Panoramic Epipolar Geometric Score) measures geometric consistency between stereo pairs on the sphere. This metric is used as a reward signal in direct to align the model outputs toward geometrically consistent stereo videos. The work is significant because it enables generative AI to produce content suitable for immersive displays like VR headsets, bypassing the need for expensive capture rigs or manual authoring.
A zero-shot pipeline that turns monocular 360° video generators into stereoscopic ones
By warping depth to produce the second eye and using a geometric reward for fine-tuning, the method outputs 4K stereoscopic 360° videos without any stereo training data.
Academic
Debabrata Mandal · Dongdong Fu · Jonathon Miller · William Villareal · Xi Peng · Praneeth Chakravarthula
UNC Chapel Hill · Dolby Laboratories
Research Digest··3 min read
In one line
A zero-shot pipeline generates 4K stereoscopic 360° video from monocular panoramic video by enforcing epipolar consistency.
Before you read this
Three ideas this paper builds onpreference optimizationA post-training method that adjusts model behavior using comparisons or self-generated preferences to improve alignment.71 papers →diffusion modelA type of generative model that learns to reverse a gradual noising process to create data.199 papers →flow matchingA generative training method that learns an ordinary differential equation mapping noise to data by matching velocity fields along probability paths.175 papers →
Read it as
The authors present EPIC, a zero-shot method to generate stereoscopic 360° videos from existing monocular panoramic video diffusion models.
The authors start with a frozen monocular panoramic video (a latent diffusion transformer trained on Internet data).
How it works
before and afterThe Daily Edition
The front page in your inbox each weekday morning: the lead, the picks, and why each one earned its place.