diffusion transformer
A diffusion transformer generates data like images by gradually removing noise from random inputs, using a transformer architecture that treats pieces of the data as tokens and pays attention to their relationships. This allows the model to handle complex patterns and create detailed, coherent outputs.
The papers focus on improving efficiency and control. They reduce computation via quantization, token pruning, or frequency-based forecasting, and enhance quality through latent compression or text-guided steering for multimodal generation.
- Papers
- 75
- Released code
- 1
- First seen
- Sept 2026
- Latest
- Nov 2026
75 papers in the last two months, against 0 in the two before.
Start here
Three papers to read first, in the order they came out- 1Image generation models can be repurposed for speech synthesis while preserving voice
ImageTTS repurposes a pretrained image diffusion transformer for speech synthesis using spectrograms as images.
Oct 2026 - 2Driving simulators follow planned paths more faithfully with overhead views
BirdDreams jointly generates perspective and bird's-eye-view videos to create reactive driving simulations from a single image.
Nov 2026 - 3Few-shot method extracts visual concepts from diffusion model activations for precise editing
Visual concept directions for image steering can be extracted from Diffusion Transformer activations using only five prompt pairs.
Nov 2026
How fast it is moving
Often studied with
flow matchingA generative training method that learns an ordinary differential equation mapping noise to data by matching velocity fields along probability paths.40 togetherdiffusion modelA type of generative model that learns to reverse a gradual noising process to create data.17 togethersparse attentionSelectively attending to only the most relevant positions or tokens to reduce computational overhead.12 togetherWho is working on it
The papers
Most central to this idea first, not most recent.
- Big Techcs.CV
Driving simulators follow planned paths more faithfully with overhead views
BirdDreams jointly generates perspective and bird's-eye-view videos to create reactive driving simulations from a single image.
NVIDIA, KAIST · Nov 2026
- Research Labcs.CV
Few-shot method extracts visual concepts from diffusion model activations for precise editing
Visual concept directions for image steering can be extracted from Diffusion Transformer activations using only five prompt pairs.
Inria Centre at Université Côte d’Azur · Nov 2026
- Big Techcs.SD
Image generation models can be repurposed for speech synthesis while preserving voice
ImageTTS repurposes a pretrained image diffusion transformer for speech synthesis using spectrograms as images.
Wuhan University, Microsoft Corporation · Oct 2026
- Academiccs.LG
Fractional dynamics model speeds up diffusion transformers without extra training
FADE accelerates Diffusion Transformers by forecasting changing internal features with a fractional-dynamics model and a precomputed lookup table, maintaining quality during longer sampling skips.
Sichuan University · Nov 2026
- Research Labcs.CV
Video AI models generate clips with 99% fewer attention computations
ESTRA accelerates video diffusion models up to 5.39× by computing only 1% of attention exactly.
The University of Sydney, CSIRO · Nov 2026
- Big Techcs.CV
AI image editors can be tricked into copying rather than editing
Maximizing attention to the reference in early DiT blocks forces editors to copy the input, preventing the requested edit.
Princeton University, Adobe Research · Oct 2026
- Big Techcs.CV
Image generators can add detail while largely preserving image structure
Flattening a token's energy distribution by scaling its channel magnitudes gives continuous control over generated image detail.
The Australian National University, Microsoft · Nov 2026
- Independentcs.RO
Video generators can control robots without producing video, cutting computing costs
Tokenizing robot actions as single video patch tokens lets a pretrained video diffusion model act as a policy without generating videos at inference.
Oct 2026
- Big Techcs.RO
Generated language gives robot policies a working memory
Jointly generating video, action, and language in one model improves robot manipulation and lets generated language serve as working memory.
Brown University, Amazon · Oct 2026
- Chinese Techcs.CV
Video generators can change characters or cameras while preserving other details
Factor-Aware Predictive Learning strengthens MLLM latent use for temporal inference and factor-aware control in cinematic video generation.
The Hong Kong University of Science and Technology, Tencent Video · Oct 2026
- Chinese Techcs.CV
Combining compression and adaptive inference for diffusion transformers without quality loss
Trajectory-aware student training with learned routing and teacher rescue makes compressed Diffusion Transformers compatible with adaptive execution, reaching FID 2.58 at 2.25× speedup.
The Hong Kong University of Science and Technology (Guangzhou), University of Electronic Science and Technology of China · Oct 2026
- Top Universitycs.CV
Unified world action model jointly learns perception, prediction, and planning
A single model can jointly reconstruct 3D hand and camera motion, predict action-conditioned video futures, and plan goal-directed manipulation trajectories, with unified training improving all three.
HKUST (GZ), Primatix · Oct 2026
- Independentcs.CV
One model combines text, image and video references for editing
OmniXRef outperforms task-specific models and commercial systems on multimodal multi-reference video editing.
Oct 2026
- Big Techcs.CV
Frequency-aware forecasting cuts diffusion transformer evaluations while preserving fidelity
SpecKCast accelerates Diffusion Transformers by forecasting DCT coefficients with Kalman filters along noise-schedule-aligned time, achieving up to 6x speedup with minimal memory.
University of Toronto, Vector Institute · Oct 2026
- Chinese Techcs.CV
Diffusion transformer depth profiles reflect prediction targets, not just feature granularity
The inverted-U depth profile in diffusion transformers reflects where the prediction target is encoded, not a coarse-to-fine semantic signal.
Sun Yat-sen University, Alibaba Group · Oct 2026
- Top Universitycs.CV
Centering activations enables calibration-free 4-bit diffusion transformer quantization
Exact per-token mean centering enables calibration-free 4-bit DiT quantization to match calibrated SVDQuant across three model families.
Tenstorrent, EPFL · Oct 2026
- Big Techcs.CV
High-fidelity latent priors improve efficient high-resolution image generation
UltraDiff distills a joint f8-f32 diffusion teacher into an f32 student, approaching f8 image quality at f32 speed.
The Hong Kong University of Science and Technology (Guangzhou), Adobe Research · Oct 2026
- Big Techcs.CV
Pixel diffusion benefits from adapting its prediction target by data direction
A direction-adaptive interpolation between clean-image and velocity prediction, plus a post-hoc convolutional refiner, improves pixel-space diffusion transformers.
University of Maryland, College Park, Adobe Research · Oct 2026
- Big Techcs.CV
Spatially addressed memory preserves scenes across long generated videos
SMART uses pose-addressed slots for bounded video memory, enabling long-horizon spatial recall without explicit 3D reconstruction.
ETH Zurich, Google · Oct 2026
- Chinese Techcs.CV
Long skip connections help diffusion transformers learn cleaner visual representations
Diffusion Transformers without long skip connections produce weaker visual features because diffusion noise pollutes the main path; adding long skip connections restores feature quality.
Institute of Information Engineering, CAS, University of Chinese Academy of Sciences · Oct 2026
- Chinese Techcs.CV
Manipulating queries and keys enables training-free MM-DiT editing
QK-Edit manipulates query-key attention in MM-DiTs for training-free text-guided image and video editing.
UC Merced, ByteDance Seed · Oct 2026
- Chinese Techcs.CV
Reusing Transformer blocks improves image generation without enlarging the model
Looped diffusion transformers with deep supervision and self-modulating attention outperform much larger models with less compute.
SenseTime Research, LeapLab, Tsinghua University · Oct 2026
- Top Universitycs.LG
Loop-specific targets let diffusion transformer improve with extra inference loops
LiFT improves ImageNet generation by repeatedly applying a shared DiT core with depth-specific trajectory targets, cutting parameters and compute versus a dense DiT baseline.
University of Amsterdam, TNO · Oct 2026
- Chinese Techcs.CV
RL-trained cache scheduler adapts video diffusion to user-specified speedups
MORCA uses latent-aware offline-to-online reinforcement learning to schedule cache reuse, improving video diffusion fidelity at user-specified acceleration targets over existing caching methods.
Shanghai Jiao Tong University, Alibaba Group · Oct 2026
- Chinese Techcs.CV
Cached token selection narrows diffusion transformers’ sparse attention quality gap
MC-Sparse achieves up to 2.32× denoising speedup over dense attention in diffusion transformers with negligible quality loss.
Fudan University, Tencent HY · Oct 2026
- Top Universitycs.CV
Text queries retrieve SAE features for steering diffusion transformers
D-Scope retrieves sparse autoencoder decoder directions for diffusion transformers by matching text queries to visual centroids of activating patches, enabling interpretable generation steering.
Pivotal Research, Mohamed bin Zayed University of Artificial Intelligence · Oct 2026
- Chinese Techcs.CV
Periodic stripe structures in video DiT attention enable efficient sparse computation
Parameterized stripe attention exploits recurring video DiT attention patterns to accelerate HunyuanVideo and Wan 2.1 by 1.57× and 1.37× with limited quality loss.
Alibaba Group · Oct 2026
- Chinese Techcs.CV
ThinkV2V uses explicit MLLM reasoning to boost complex video editing
ThinkV2V improves instruction-guided video editing by making MLLMs reason explicitly before generating edits.
The Chinese University of Hong Kong, Zhejiang University · Oct 2026
- Industrycs.CVcode
Training-free wrapper accelerates causal video diffusion to 50 FPS without quality loss
UnStep accelerates causal video diffusion models to 50 FPS on an H100 without quality loss and without retraining.
Meta Superintelligence Labs · Oct 2026
- Big Techcs.CV
Variable-size tokens cut diffusion costs where less detail is needed
LoT Diffusion enables image and video generation with fewer tokens in low-detail regions, achieving up to 2.04x and 3.53x speedups with quality preserved.
Stanford University, Google · Oct 2026
- Chinese Techcs.CV
Adaptive reference attention reuse speeds up conditioned diffusion transformers
RefAdapt-DiT adaptively reuses reference computations in diffusion transformers, achieving up to 3.54x speedup on Qwen Image Edit with comparable quality.
Tencent · Oct 2026
- Chinese Techcs.CV
Stochastic attention routing enables efficient RL alignment for diffusion models
Routing Policy Optimization uses temporary stochastic attention routes to improve reward-aligned visual generation with less training compute while preserving dense deterministic inference.
National University of Singapore, ByteDance · Oct 2026
- Chinese Techcs.CV
Interaction-aware sparse attention and timestep-adaptive sparsity accelerate video generation
iCATS achieves state-of-the-art efficiency-quality trade-off in video generation through interaction-aware sparse attention and SNR-guided sparsity.
Hangzhou Normal University, T-Head Semiconductor Co., Ltd. · Oct 2026
- Top Universitycs.CV
Structured residual connections improve diffusion transformers by enabling adaptive cross-layer information retrieval
Structured residual connectivity in Diffusion Transformers yields faster convergence and better FID with negligible added parameters.
National University of Singapore, Shanghai Jiao Tong University · Oct 2026
- Industrycs.CV
Kandinsky 6.0 jointly generates short videos with synchronized sound
Kandinsky 6.0 Video generates synchronized 5-second video and 44kHz audio, including lip-sync, from text or image prompts.
Kandinsky Lab · Oct 2026
- Big Tech
Reciprocal attention regularization improves alignment in joint video generation
Joint multimodal generators have asymmetric cross-attention; RecCAR aligns weaker modality-to-video attention to improve consistency.
Bar-Ilan University, NVIDIA · Sept 2026
- Chinese Techcs.CV
Spatial gradient maps improve RL alignment of flow-based image models
SGA-Flow-GRPO uses spatial gradient-guided credit assignment to achieve state-of-the-art alignment quality for flow-based generative models on GenEval.
Sun Yat-Sen University, Baidu · Sept 2026
- Chinese Techcs.AI
AI video generator uses separate visual and audio timelines to control cinematic dialogue
Découpage independently controls audio and visual timing for long multi-shot video generation, enabling cinematic coherence.
The Hong Kong University of Science and Technology, Ant Group · Nov 2026
- Industrycs.CV
Image generators can improve their own samples without external judges
A training recipe, Scout-Refine-Correct, lets flow-matching image generators refine their own samples during inference, matching an oracle-based best-of-N baseline on ImageNet without a verifier.
FusionBrain, AXXX, HSE Unifersity · Nov 2026
- Top Universitycs.CV
Streaming video generators can now output frames at near-watchable speed
Streaming video generation can emit one finished chunk per concurrent forward round by pipelining denoising stages across model replicas, preserving quality while speeding throughput.
Shanghai Jiao Tong University, Xi’an Jiaotong University · Nov 2026
- Academiccs.CV
Generating video directly in pixels can beat models that compress first
Generating video directly in pixel space, without a pretrained autoencoder, beats latent-space models when the transformer's patch size is varied with network depth.
FAIR, Meta, Sorbonne University, Paris · Nov 2026
- Top Universitycs.CV
Video generators can run twice as fast without retraining
Accurate Block Attention (AB-Attention) skips 89-92% of attention in video diffusion transformers while improving PSNR by up to 4.4 dB.
InfRec, The Chinese University of Hong Kong, Shenzhen · Oct 2026
- Chinese Techcs.CV
Jointly generating object shapes and camera poses improves sparse-view reconstruction
A native 3D diffusion model jointly recovers object shape and camera motion from a few uncalibrated views.
The University of Hong Kong, Transengram · Oct 2026
- Industrycs.CV
Parallel adapter composition yields real-time joint audio-video generation without chained training
Parallel adapter composition achieves real-time streaming audio-video generation by adding separately trained causal and few-step adapters.
LIGHTSPEED, Independent Researcher · Oct 2026
- Chinese Techcs.LG
H-SVDQuant improves 4-bit quantization of diffusion transformers with low-rank branches
H-SVDQuant achieves rank-4 W4A4 quantization that surpasses rank-32 SVDQuant on diffusion transformers and LLMs.
The University of Hong Kong, Huawei Technologies Co., Ltd. · Oct 2026
- Chinese Techcs.CV
Diffusion models benefit most from teacher features they still lack
RARE selects the teacher layer with the largest recoverability gap, improving FID and training speed.
Central South University, The Hong Kong University of Science and Technology · Oct 2026
- Top Universitycs.CV
Touch signals improve predictions of how hands manipulate objects
A world model conditioned on tactile signals reduces hand motion underestimation from 23% to 9%.
Yale University, Texas A&M University · Oct 2026
- Chinese Techcs.CV
Image generators create less distorted panoramas with fewer visible seams
CubeAtlas representation enables distortion-free and seamless 360-degree panorama generation with better quality and lower attention computation than equirectangular projection.
Tencent · Oct 2026
- Top Universitycs.LG
Diffusion reward models capture multimodal preference that scalar scores miss
DRM models reward as a multimodal conditional density, matching or surpassing baseline reward models.
Tsinghua University, The Chinese University of Hong Kong · Oct 2026
- Chinese Techcs.CV
High-fidelity 3D texture models can train on images alone
3D texture generation can be trained using only 2D images, without any 3D assets.
MMLab, CUHK, Tencent Hunyuan · Oct 2026
- Big Techcs.CV
Diffusion in texture space generates high-quality 3D materials from text or photos
Generating high quality 3D object materials entirely in texture space avoids view consistency problems.
NVIDIA · Oct 2026
- Chinese Techcs.CV
Fine-grained routing makes sparse video attention faster without quality loss
VSA2 makes video DiT attention 8.9x faster with comparable quality by using a fine-grained router and adaptive sparsity.
University of California, San Diego, ByteDance Seed · Oct 2026
- Chinese Techcs.CV
One-step diffusion framework restores 1080p video in real time with efficient streaming attention
FastVR restores high-resolution video at 11 FPS using a one-step diffusion model with a lightweight VAE and chunk-wise causal attention.
Alibaba Group, Xidian University · Oct 2026
- Chinese Techcs.CV
Sparse flow matching generates 3D shapes at ultra-high voxel resolution
Filigree3D generates detailed 3D geometry from one image at 2048^3 voxel resolution by combining sparse computation, local-global attention, and multi-scale visual conditioning.
Alibaba Group · Oct 2026
- Big Techcs.RO
Robots learn better by predicting changes rather than whole scenes
Learning only the visual change between frames, not the full future, improves robot policy success and efficiency.
Michigan State University, Cisco · Nov 2026
- Top Universitycs.CV
Lifting latent video models to pixels enables higher resolution and per-frame control
PixVid converts a pretrained latent video diffusion model to generate frames directly in pixel space, improving resolution, control, and latency.
Roblox, University of California, Merced · Nov 2026
- Top Universitycs.CV
High-resolution image editing gets a 4x speedup by keeping a low-resolution anchor
FLASH enables faithful 2K/4K image editing by anchoring a high-resolution detail branch to a frozen 1K editing backbone.
Shanghai Innovation Institute, Fudan University · Nov 2026
- Academiccs.CV
Tuning only the first denoising step restores lost diversity in distilled models
Diversity loss in distilled diffusion models is concentrated in the first step and can be recovered by a lightweight first-step bias adapter.
Graz University of Technology, Complexity Science Hub Austria · Oct 2026
- Chinese Techcs.CV
Image-to-3D method fixes orientation and faithfulness in coarse generation
Coarse3D generates orientation-controllable and faithful sparse 3D structures from arbitrary unposed images.
Alibaba Group · Nov 2026
- Chinese Techcs.CV
Plane primitives serve as compact persistent memory for video world models
Plane primitives can serve as a compact persistent 3D memory for video world models, enabling coherent long-context generation.
Tsinghua University, Ant Group · Oct 2026
- Top Universitycs.CV
One model edits people, voices and movements in sync
AVE unifies mask-free human-centric audio-visual editing for person swap, insert, delete with superior synchronization.
University of Hong Kong, National University of Defense Technology · Oct 2026
- Big Techcs.AI
Explicit predictive states boost physical plausibility in video world models
Physical Registry adds explicit predictive states to video DiTs, improving physical plausibility with minimal parameters.
Yonsei University, Microsoft Research Asia · Oct 2026
- Chinese Techcs.CV
Joint inverse and forward rendering produces more coherent intrinsic scene maps
Jointly modeling intrinsic channels and closing the inverse-forward rendering loop with a dual cycle-consistent objective improves intrinsic decomposition and forward rendering coherence.
Shenzhen University, Robotics X Lab, Tencent · Oct 2026
- Chinese Techcs.CV
World model forecasts future 3D geometry directly without recursive rollouts
GeoWM directly predicts future 3D geometry at requested horizons, improving accuracy and cutting long-horizon inference time by avoiding recursive rollouts.
Huawei Noah's Ark Lab · Oct 2026
- Big Techcs.CV
Adaptive tokenization cuts compute for pixel-space diffusion models by up to 75% with no fidelity loss.
BudgetPix matches full-compute image fidelity using 25% of the compute by adaptively tokenizing based on visual complexity.
University of Illinois Urbana-Champaign, Google · Oct 2026
- Big Techcs.CV
Observation-only videos can directly pretrain robot action policies
NAVA-WAM directly pretrains an action policy from observation-only videos using future-video flow matching and outperforms prior world action models.
Meta AI, University of Copenhagen · Oct 2026
- Chinese Techcs.CV
Image-based visual examples enable powerful video editing without large paired video datasets
Video editing can be achieved by routing edits through the image domain with an in-context visual demonstration, reducing the need for expensive paired video data.
National University of Singapore, ByteDance Seed · Oct 2026
- Big Techcs.CV
Last-frame layouts guide video generation through large camera moves
LIFT enables specifying layout of future views in video generation via on-policy self-distillation from a dense-layout teacher, improving controllability under large viewpoint changes.
UC San Diego, University of Virginia · Oct 2026
- Academiccs.CV
Predicting motion, not video, improves robot learning from human footage
Predicting 3D scene flow instead of video from pretrained models improves robot policy learning.
Georgia Institute of Technology · Nov 2026
- Chinese Techcs.CV
Image generators can train over 140 times faster with pixel feedback
SpeeDiff enables end-to-end training of latent diffusion models using a Tweedie loss that prevents latent collapse, achieving dramatically faster convergence.
ByteDance Seed, California Institute of Technology · Oct 2026
- Independentcs.CV
Position-based self-conditioning improves single-stage 3D generation without extra stages
Position Forcing improves single-stage 3D generation by using recovered token positions as progressive spatial guidance during denoising.
Oct 2026
- Chinese Techcs.CV
Hierarchical communication helps video world models handle more players
Hierarchical player-group-global communication improves multiplayer video consistency and control while scaling to more players with less attention computation.
The University of Hong Kong, ARC Lab, Tencent IEG · Oct 2026
- Top Universitycs.CV
Predicting velocity fields helps video models generate more realistic fluids
FluidGen generates physically plausible fluid videos by jointly predicting a 3D velocity field alongside RGB.
Stanford University · Oct 2026
- Chinese Techcs.CV
Velocity corrections improve instruction following in few-step text-to-image generation
A sub-20M-parameter velocity residual corrector improves few-step text-to-image instruction following by 17.60 percentage points without altering the frozen generator or adding forward passes.
The University of Osaka, Alibaba Group · Oct 2026
- Big Techcs.LG
Denoiser Jacobian spectrum links to generative model quality and enables targeted improvements
Better generative denoisers have larger principal Jacobian eigenvalues, and regularizing those responses while suppressing noisy directions improves generation quality.
Stony Brook University, Microsoft Research · Oct 2026
Concepts are extracted from each paper and reused across the corpus, so this page grows on its own as the desk reads.