Generative media

Generating images, video, 3D and audio: diffusion, flow matching, editing, generative pipelines

273 articles · page 1 of 6

Generative media

Few-shot method extracts visual concepts from diffusion model activations for precise editing

Quignon and Dantcheva show that visual concept directions can be extracted directly from the internal activations of Diffusion Transformers using as few as five positive and negative prompt pairs. Their training-free framework, combining global and masked difference-of-means estimators, enables continuous steering of both scene-level concepts (lighting, weather) and localized attributes (object color, facial expressions) at inference time, outperforming text-encoder steering baselines and rivaling specialized editing models.

·3 min
Generative media

AI image generators learn to follow dense prompts by using their own visual understanding as a teacher

The authors propose Structural Self-Teaching (SST), a framework that improves compositional image generation in unified multimodal models (UMMs) by leveraging the model's own understanding path as a source of structural supervision. SST extracts phrase-conditioned support masks from internal attention maps in the understanding path and uses them to align generated phrase regions with target regions and to modulate intermediate spatial states during training, all without requiring external annotations or inference-time cost.

·2 min
Generative media

Bottlenecked diffusion models predict best when targeting clean data, not noise

The authors systematically analyze how the choice of prediction target and loss space affects denoising diffusion models, both theoretically and empirically. They show that under a rank-deficient bottleneck, noise and velocity prediction incur an irreducible squared-error floor that clean-data prediction avoids. Experiments across CIFAR-10, Imagenette, and ImageNet-256 confirm that noise-prediction errors match the theoretical bound, while clean-data prediction exploits low-dimensional data structure.

·2 min
Generative media

Matching final outputs improves few-step image generator distillation

The authors propose eDMD, a distillation method that trains a fake score model solely on the student's final generated outputs rather than on intermediate states. This ensures the distribution matching signal vanishes exactly when the student's final output distribution matches the teacher's, removing a spurious gradient component. Experiments on four large-scale image generators show consistent improvements over prior distribution matching approaches for 4-step generation.

·3 min
Generative media

Lifting latent video models to pixels enables higher resolution and per-frame control

The authors present PixVid, a method that takes a pretrained latent video diffusion model and adapts it to generate every frame in pixel space. By predicting clean video rather than velocity and using a larger noise shift, the finetuned model matches the latent baseline in quality within 1.3 million training clips while enabling 2× higher resolution at the same per-frame token count, maintaining 98% of source motion during depth-conditioned editing (versus 83% for the latent baseline), and reducing control-to-screen latency from 495 ms to 34 ms.

·3 min
Generative media

Code simulations teach video generators realistic physics with little data

The authors introduce PhysOPSD, a method that uses executable code as privileged context to improve the physical plausibility of videos generated by diffusion models. By converting real-world scenes into simulation programs and using them to provide dense, on-policy corrective feedback via self-distillation, the method achieves better physics than supervised fine-tuning or reinforcement learning baselines, using only 3,000 training samples.

·3 min
Generative media

Fractional dynamics model speeds up diffusion transformers without extra training

The authors find that features in diffusion transformers evolve with anomalous diffusion, not standard Brownian motion. They propose a training-free acceleration framework that uses a fractional calculus operator to better predict these dynamics, enabling larger skips between sampled timesteps. The method achieves state-of-the-art speed-ups on image, video, and audio models with negligible overhead.

·1 min
Generative media

Diffusion model trajectories follow a universal geometric shape

The authors show that the deterministic sampling trajectories of diffusion models, the paths from noise to data, take approximately the same geometric shape regardless of the data type (images, audio, text, or video) or the specific model architecture. They demonstrate that this universal structure is well described by a hyperbolic curve that arises from an exactly solvable model based on the optimal linear denoiser.

·3 min
Generative media

Persistent memory and reinforcement learning reduce drift in long driving simulations

WAM-Sim combines a persistent hierarchical memory (structural, scene, instance tokens) with an online flow-matching reinforcement learning framework to prevent structural drift, identity mutations, and quality decay in long-horizon driving simulations. On the nuScenes benchmark, it achieves FID=3.47 and strong downstream perception; on self-collected minute-scale data, it maintains high scene and identity coherence.

·2 min
Generative media

Video generators can nearly halve history storage while improving reconstruction quality

Lu and colleagues present NanoKV, a way to compress the stored history that autoregressive video models use to maintain scene consistency. On HY-WorldPlay, they report 26.09-fold effective compression, nearly twice that of the competing KVTC method, alongside better reconstruction quality. An optional faster version also reduces generation time without retraining the underlying video model.

·2 min
Generative media

High-resolution image editing gets a 4x speedup by keeping a low-resolution anchor

The authors present FLASH, an image editor that natively operates at 2K/4K by splitting the task: a frozen 1K branch anchors the instruction-conditioned edit, while a high-resolution branch adds local detail using sparse attention. On a high-resolution editing benchmark it achieves the strongest structural fidelity among compared methods, and a 4K forward run is about 4 times faster than dense attention.

·2 min
Generative media

A frozen AI model can imitate an unseen dataset from one example

Li, Shao, Zhou and colleagues formalise single-reference zero-shot time series generation: a model is pretrained once, then shown one sequence from an unseen domain and asked to generate more like it, without updating any parameters. Their system, MarkovTS, encodes local shapes and the way those shapes follow one another, and uses that encoding to steer a flow-matching generator. Across 12 unseen datasets it matched or beat strong baselines on most distributional comparisons, and its synthetic samples improved forecasting, classification and anomaly detection when used as training data.

·3 min
Generative media

Streaming video generators can now output frames at near-watchable speed

The authors introduce Wave Forcing, a co-designed generation, training, and execution framework that enables continuous, multi-step video generation at speed-of-light rates. By staggering denoising and cache-refresh stages across full-model replicas and using causal chunk-wise attention, they achieve one output chunk per concurrent forward round after pipeline fill. On eight H200 GPUs, 4-step Wave Forcing reaches 134.8 end-to-end frames per second, a 7x speedup over single-GPU baselines, while maintaining video quality comparable to state-of-the-art rolling methods.

·3 min
Generative media

Image-to-3D method fixes orientation and faithfulness in coarse generation

The authors present Coarse3D, a sparse-structure generation model that produces orientation-controllable and structurally faithful voxel geometry from arbitrary unposed images. It introduces a first-frame anchored canonical space to eliminate ambiguous semantic canonicalization, and a dual-tower DiT with spatial-prior guided attention to improve 2D-3D grounding, substantially improving coarse-to-fine 3D generation pipelines.

·2 min