Tex-Zero converts high-quality 2D images into 3D training samples by representing each image as a plane in 3D space and applying random patch-wise rotations and aggregation. This constructed data is used to train a VAE and DiT for native 3D texture generation. Despite never seeing real 3D assets during training, the VAE reconstructs real 3D assets with high quality, and the DiT generates high-fidelity textures conditioned on multi-view images. This finding challenges the common belief that 3D assets are necessary, offering a scalable data paradigm using abundant 2D images.
High-fidelity 3D texture models can train on images alone
Tex-Zero replaces real textured 3D assets with image patches arranged into synthetic spatial structures.
Chinese Tech
Jiangshan Wang · Zeqiang Lai · Jiayi Guo · Xin Yang · Xin Huang · Jiarui Chen · +3 more
MMLab, CUHK · Tencent Hunyuan · Tsinghua University · Fudan University · Shanghai Innovation Institute
Research Digest··2 min read
In one line
3D texture generation can be trained using only 2D images, without any 3D assets.
Before you read this
Two ideas this paper builds ondiffusion transformerA neural network architecture combining transformer layers with diffusion processes for high-quality image and video generation.75 papers →flow matchingA generative training method that learns an ordinary differential equation mapping noise to data by matching velocity fields along probability paths.173 papers →
Read it as
Wang and colleagues present Tex-Zero, a native 3D texture system trained exclusively on 2D images rather than textured 3D assets.
The authors treat images and textured surfaces as related data structures: each associates colors with spatial locations.
How it works
before and afterThe Daily Edition
The front page in your inbox each weekday morning: the lead, the picks, and why each one earned its place.