News without the noise
Generating images, video, 3D and audio: diffusion, flow matching, editing, generative pipelines
43 articles
The authors propose RefAdapt-DiT, a training-free framework that adaptively controls joint attention computation between reference images and target generation in diffusion transformers. By monitoring reference drift and target-to-reference attention, it selectively reuses stale reference states, achieving speedups of up to 3.54× on image editing with minimal quality loss.
Xu and Wang train a few-step image generator by first imitating a pretrained teacher, then optimizing the entire sequence of latent transitions against real ImageNet data. Their three-stage model reports FID 1.11 and Inception Score 282, while training entirely in latent space and without directly minimizing Fréchet distance.
Zhang et al. introduce VSA2, a trainable sparse-attention system for video diffusion transformers that selects a small, query-dependent subset of token interactions. The authors report substantially faster 720p generation than a FlashAttention-3 baseline, while maintaining comparable or better video quality.
Wang and colleagues present Tex-Zero, a native 3D texture system trained exclusively on 2D images rather than textured 3D assets. They convert image patches into artificial 3D arrangements, then use them to train both a variational autoencoder and a diffusion transformer. The authors report that the resulting model reconstructs and generates detailed textures on real, previously unseen 3D geometry.
Wang et al. identify inaccurate critic estimates as a source of oversaturation and artifacts during Distribution Matching Distillation, a method for compressing diffusion generation into a few denoising evaluations. Their proposed projection filters the error-prone component of each update, improving stability and benchmark performance in four-step video and joint video-audio generation.
Meng et al. propose CoDrive, a cross-vehicle video generation framework that produces world-consistent observations for multiple vehicles in the same scene. By interleaving local and global self-attention and injecting camera trajectories in a shared coordinate system, the model significantly improves trajectory controllability and cross-agent geometric and instance consistency while maintaining visual quality.
Wang and colleagues study how to choose the teacher layer and duration of representation alignment when training diffusion transformers. They find that alignment is most useful for features the student cannot already reconstruct, then use this principle to improve image generation while reducing training cost.
Zhang, Shi, Liu and colleagues present a common mathematical account of distributional training, where generators learn by matching collections of real and generated features from frozen image encoders. Their resulting method, MGFlow, represents those distributions as Gaussian mixtures and reports stronger one-step generation than prior feature-distribution objectives on ImageNet and text-to-image benchmarks.
Deschenaux et al. introduce Simplex Diffusion Models, which keep intermediate predictions as probability distributions rather than repeatedly collapsing them into categorical token choices. The authors derive closed-form reverse transitions, use ordinary cross-entropy training, and report gains over masked and uniform diffusion on code generation, including after aggressive sampling distillation.
Wang and colleagues formulate feed-forward reconstruction and conditional generation as two inference modes of one flow-based model. A single step predicts the conditional mean, while multiple denoising steps produce sharper solutions on the learned data distribution, with accuracy depending on how strongly the input observations constrain the answer.
The authors present UnStep, a training-free wrapper that accelerates already-distilled causal video diffusion models by running them with fewer denoising steps and a limited temporal KV cache. Two inference-only mechanisms—clean-cache refinement (renoise and refine) and truncated SVD on attention projections—recover the quality lost by these reductions, while an optimized runtime stack further boosts throughput. UnStep achieves 50 frames per second on a single H100 GPU and 77 FPS on a GB200 GPU without quality degradation, all without retraining.
The authors propose OREO, a framework that enhances visual fidelity of 3D generators by creating a dynamic optimization loop. They use a 2D image editing model to refine rendered views of 3D outputs on the fly, producing high-quality pseudo-targets. Then, a latent contrastive objective distills the improvements back into the 3D generator, resulting in significantly more realistic assets.
The authors introduce a flow-matching generative model whose velocity field is parameterized by a permutation-equivariant Graph Meta Network. This allows them to learn directly from collections of independently trained neural networks without the need for approximate neuron alignment. The method generates new networks that closely reproduce the joint statistics of accuracy, functional similarity, and weight similarity observed in training collections, and a single conditional model can produce task-specific networks for heterogeneous architectures and previously unseen hidden-width configurations.
Chen and colleagues survey how autoregressive video generators can preserve relevant history after it leaves their limited context windows. They argue that effective memory requires more than storing information: retained states must remain accurate, retrievable, and causally influential during later generation.
The authors present a framework for generating multi-step GUI interaction data without installing or executing the software being depicted. Fine-tuning Qwen3.5-35B-A3B on the resulting synthetic trajectories raised its OSWorld mean task score from 33.0% to 40.8% and more than doubled its ScienceBoard success rate.
Peng et al. identify a mismatch in joint video and action models: training perturbs recorded actions while leaving the corresponding video tied to what originally happened. CtrlWAM instead executes perturbed actions in a simulator and trains on the resulting counterfactual visuals, producing futures that better reflect predicted or commanded behavior.
The authors present FaV-A, a multimodal agent that generates positive-negative space images through three connected stages. It first creates a base object, then analyzes its contours to propose secondary semantics for the negative space, and finally issues compositional instructions for the image generation model. In user studies and ablations, FaV-A produced more visually coherent and semantically aligned compositions than zero-shot MLLM baselines.
Zhan et al. introduce a dataset and evaluator for diagnosing incorrect object motion in generated videos, rather than judging only appearance or prompt alignment. Their evaluator converts videos into motion-aware representations and produces human-aligned scores across three dimensions, alongside explanations tied to specific failures.
Wang, Wang and Jin introduce an agentic video pipeline that represents each story event as a structured change to characters, objects and locations rather than relying primarily on prior generated frames. On a new long-form storytelling benchmark, the authors report that StoryEngine outperforms Direct I2V, ViMax and MovieAgent across narrative, cross-shot coherence and visual consistency evaluations.
Zhao et al. present a harness that turns code-based visual generation into an iterative process of construction, rendering, inspection, and revision. Across image and video benchmarks, the best evaluated model completed every task successfully, while meeting all specified quality thresholds on 96.0% of image tasks and 76.9% of video tasks.
Rahamim et al. identify a directional imbalance in multimodal diffusion transformers: motion or audio attends strongly to video, while exerting comparatively weak influence over video generation. Their regularizer narrows this correspondence gap, improving anatomical consistency and audio synchronization without adding an inference-time component.
Xu et al. investigate why standard Distribution Matching Distillation produces visually plausible but nearly static rollouts when applied to video world models. Their DyMD framework adjusts teacher and critic supervision around motion, improving interaction dynamics and downstream planning while retaining a four-step, 1.3-billion-parameter student.
Huang et al. developed a learned physical simulator that jointly refines an entire prediction window while preserving a structured, local model of motion. On two HamiBalls benchmarks containing smooth dynamics and collisions, it produced lower phase-space errors than similarly sized transformer and diffusion baselines.
Du et al. address a trade-off in representation autoencoders: shallow visual features preserve pixel detail, while deeper features can support better image generation. On ImageNet-256, their randomized layer-fusion training improves reconstruction flexibility and reduces unguided generative FID by up to 29%, without modifying the pretrained DINOv3-L encoder.
Zhu et al. trained a 397-billion-parameter prompt enhancer on 1.05 million real-world videos using prompts reconstructed from video content rather than conventional forward rewriting. In roughly 11,000 blind pairwise assessments, the resulting plans improved preference for Wan3.0 outputs, with the largest reported gain occurring on 30-second videos.
Fang and colleagues present CoDeR, a world-modeling framework that represents rules, entities, and state explicitly in code rather than only through generated video. The authors report that this separation supports persistent memory, open-ended interaction, autonomous evolution, and multiple interacting agents beyond the current visual observation.
The authors propose CompAdapt, a diffusion-based text-to-video framework that improves physical consistency by modeling composite motions beyond simple single-type dynamics. The system translates natural language prompts into structured physical semantics, uses a dynamics-aware prior matching module for one-shot adaptation to novel environments, and employs a physics-aware latent feature fusion to maintain visual fidelity. On physics-focused benchmarks, CompAdapt outperforms both general T2V models and previous physics-constrained methods in physical plausibility while preserving high visual quality.
The authors propose a guided super-resolution approach for digital surface models (DSMs) using denoising diffusion, improving coarse 5 m DSMs to 0.5 m resolution by leveraging high-resolution optical imagery. Experiments on several Central European cities show the method produces DSMs with crisper building outlines and more detailed roof structures than conventional interpolation or filtering.
The authors systematically benchmark three approaches to identity preservation in generative image models: encoding identity in the input context, using trainable subject-specific parameters (like LoRA), or maintaining a persistent identity layer. Their results show that persistent identity layers consistently reduce identity degradation across iterative edits, small subject scales, and multi-subject compositions, while preserving image quality and instruction adherence.
The authors trained a 6B diffusion transformer from scratch, first using image-only pretraining and mid-training to establish a visual generative prior before relying more heavily on image-text supervision. They report leading Qwen-Image-Bench scores among open-source models in both English and Chinese, and release the model weights, code, and training recipes.
Chen et al. turn MiniMax-H3 into an interactive world model by expressing actions as paired character and camera instructions aligned with video time steps. Their lightweight adaptation uses 8,000 gameplay samples and updates 0.199% of the model’s parameters while retaining generation quality and extending control to unseen scenarios.
Xu et al. developed a group-image generation framework that represents each requested person with an addressed identity token, predicts an identity–layout plan, and uses that plan to guide synthesis. On an identity-disjoint benchmark, the method improved face similarity over GPT-Image-2 while covering 97.3% of requested identities with a 2.8% duplicate rate.
The authors propose WorldWeaver, a streaming multi-agent video diffusion model that augments autoregressive rollout with cross-agent world state registers: learnable tokens that maintain shared world information, track individual agent status, and are updated after each generated chunk. The registers are grounded with supervision from agent status, global bird's-eye views, and scene text. Experiments in two-agent Minecraft show that explicit world-state modeling improves logical consistency and generation quality over baselines that only carry forward observation history.
The authors introduce SANA-Video 2.0, a video diffusion transformer that uses a hybrid of linear and softmax attention to achieve efficient long-sequence processing. The model, scaled to 5B and 14B parameters, generates 720p video on a single H100 GPU and achieves a VBench score of 84.30 at 480p in 13.2 seconds, matching the quality of larger softmax-based models while being significantly faster.
Liu et al. present SimWorlds, a multi-agent framework that generates dynamic, editable 4D scenes from natural language descriptions. The system outperforms prior baselines in visual fidelity and physical consistency on a new benchmark, 4DBuildBench.
The authors introduce COMFYCLAW, an agentic harness that controls ComfyUI workflows for image generation. By representing workflow construction as typed graph editing, reverting invalid edits, and using a region-level vision-language model (VLM) verifier to provide actionable repair suggestions, COMFYCLAW evolves a skill library from past trajectories, errors, and verifier feedback. Across four benchmark splits, three agent models, and two image backbones, COMFYCLAW achieves the best average evaluation score in all six agent configurations, and human annotators prefer its outputs over variants without skill evolution.
The authors propose Qwen-Image-Agent to address the context gap where user requests for image generation are often underspecified, implicit, or depend on up-to-date knowledge. The framework uses Context-Aware Planning to identify missing details and Context Grounding to acquire them from reasoning, search, memory, and feedback. On the newly introduced IA-Bench benchmark and two other datasets, it achieves state-of-the-art performance against strong baselines.
The authors present Sol Video Inference Engine, a training-free framework that uses parallel agent modules to optimize cache, sparse attention, token pruning, quantization, and kernel fusion for a given model-hardware-configuration triplet. Across three models (64B Cosmos3-Super, 22B LTX-2.3, 2B SANA-Video), the full stack achieves over 2x end-to-end acceleration while maintaining near-lossless VBench quality, with minimal human effort.
The authors propose SceneConductor, a multi-agent framework that generates complete 3D scenes from a single input image. It decomposes the task into three stages—scene initialization, environment construction, and multi-agent refinement—and introduces a geometry-aware layout predictor trained with sparse point-map priors. The method outperforms prior approaches on geometric accuracy, spatial consistency, and perceptual realism across benchmark datasets.
The authors introduce ViMax, an agentic video generation framework that coordinates multiple specialized agents to produce long-form videos with narrative planning and visual consistency. By combining a hierarchical narrative engine with retrieval-augmented generation and dependency-aware visual tracking, ViMax maintains global story coherence and consistent character/environment states across scenes, addressing limitations of existing short-clip methods.
The authors propose GenClaw, an agentic image generation framework that breaks generation into three stages: conceptualization, code-based sketching (e.g., SVG, HTML, Three.js), and final texturing via an image model. This replaces black-box prompt rewriting with explicit, editable code intermediates, enabling precise control and interpretability.
The authors propose Co-Director, a hierarchical multi-agent framework that formalizes video storytelling as a global optimization problem. They introduce a multi-armed bandit for global creative direction and a local multimodal self-refinement loop for identity and consistency. GenAD-Bench, a 400-scenario dataset, shows significant improvements over baselines.
The authors propose VisionCreator, a native visual-generation agentic model that unifies Understanding, Thinking, Planning, and Creation (UTPC) capabilities. End-to-end trained on a new dataset and optimized via Progressive Specialization Training and Virtual Reinforcement Learning, VisionCreator-8B/32B models outperform larger closed-source models on the new VisGenBench benchmark across multiple dimensions.