Unified diffusion LLM learns from AR and flow models via cross-paradigm distillation

NLD-Uni introduces a self-refining masked diffusion process and on-policy distillation from autoregressive and continuous-flow teachers, closing the gap to specialized models.

Industry

Shufan Li · Hanrong Ye · Greg Heinrich · Yonggan Fu · Haotian Zhang · Sifei Liu · +2 more

Research Digest·28 October 2026·3 min read

Illustration: Unified diffusion LLM learns from AR and flow models via cross-paradigm distillation

In one line

A unified discrete diffusion LLM can work for both understanding and generating images, using self-refinement and cross-paradigm distillation to match specialized models.

Before you read this

Three ideas this paper builds on

diffusion language modelA language model that generates text by iteratively denoising a latent or discrete representation.27 papers →unified multimodal understanding and generationA single end-to-end model processes and generates multiple data types (e.g., images, text) with equal competence.22 papers →on-policy distillationA training method where a student model learns from a teacher using its own current policy's outputs, often improving reasoning and instruction following.100 papers →

Read it as

Understanding images and generating them normally use different engines, since one reads tokens one by one and the other paints a continuous flow. NLD-Uni is a unified large language model (LLM) handling both with one mechanism.

It uses discrete diffusion, corrupting tokens during training and learning to restore them, like erasing and refilling a mosaic. It distills knowledge from stronger teachers of different kinds, called cross-paradigm , and replaces one big jump from blank to final token with many small revision steps.

On tests of visual understanding, grounding, text-to-image generation, and image editing, NLD-Uni leads among unified diffusion LLMs and closes much of the gap to specialized systems. A single unified model can now approach dedicated toolkits.

How it works

before and after

RSS

The Daily Edition

The front page in your inbox each weekday morning: the lead, the picks, and why each one earned its place.

Concepts

diffusion language model unified multimodal understanding and generation on-policy distillation knowledge distillation self-supervised learning