The authors conditioned a video diffusion model on a single identity image, a learned facial-expression code and renderings of a proxy head-and-torso mesh.
A controllable video model can serve as a photorealistic 3D avatar
The system animates a person from one image while independently controlling expression, upper-body pose and camera viewpoint.
Academic
Simon Giebenhain · Tobias Kirschstein · Liam Schoneveld · Davide Davoli · Zhe Chen · Matthias Nießner
Technical University of Munich · Woven by Toyota · Toyota Motor Europe
Research Digest··2 min read
Giebenhain and colleagues fine-tuned a video diffusion model to act directly as an avatar, rather than generating or rendering an explicit 3D asset.
§