Variable-size tokens cut diffusion costs where less detail is needed
Nakayama et al. introduce Level-of-Token Diffusion, which assigns smaller tokens to detail-sensitive regions and larger tokens to simpler areas such as skies, flat surfaces or blurred backgrounds. By shortening the sequence processed by a diffusion transformer while retaining full-resolution denoising updates, the method achieved reported average speedups of up to 2.04 times for images and 3.53 times for videos.
