The authors represented images and videos with multiresolution layouts in which each token covers a rectangular patch of variable height and width.
Variable-size tokens cut diffusion costs where less detail is needed
LoT Diffusion adapts pretrained image and video models to process coarse or fine spatial tokens according to a planned detail layout.
Big Tech
Kiyohiro Nakayama · Brian Chao · Jan Ackermann · Hansheng Chen · Federico Tombari · Leonidas Guibas · +2 more
Stanford University · Google
Research Digest··2 min read
Nakayama et al.
Why this paper
From Google and Stanford University
In one line
LoT Diffusion enables image and video generation with fewer tokens in low-detail regions, achieving up to 2.04x and 3.53x speedups with quality preserved.
What we could check
- ·No code link found
- ·No weights link found
- ·No dataset link found
- ·No compute details found
- ✓Limitations stated by the authors (2 noted)
- ·No benchmark numbers found
Observed from the paper text and links we have. Absence here means we did not find it, not that it does not exist.
§