Creating realistic 3D objects for virtual worlds is hard. Current methods either produce meshes with simple materials or fix the lighting, making it hard to change later.
VideoRNG uses a two-stage process. First, a video model generates a rotating view of an object from a text description or image. Second, a reconstruction model turns frames of that video into millions of tiny Gaussian shapes with neural features that respond to light, enabling ray tracing for relighting.
The method creates complex objects like fur and spider webs that can be relit and placed in new scenes. Using depth anchoring improved image quality, with a measure called peak signal-to-noise ratio rising from 25.39 to 29.22.
