AI models can improve image understanding and generation from their outputs

Zhang and colleagues report gains on two multimodal models by using reliable portions of self-generated text and image sequences to guide further training.

Chinese Tech

Peng Zhang · Qian Chen · Xianyin Zhang · Yimin Tian · Ruite Fei · Dongyi Lv · +3 more

Xi’an Jiaotong University · Qwen DianJin Team, Alibaba Cloud Computing

Research Digest·5 November 2026·2 min read

Illustration: AI models can improve image understanding and generation from their outputs

In one line

Deciding per sample whether text or image is the more reliable half of a self-generated multimodal trajectory lets a unified model improve both understanding and generation without outside supervision.

Before you read this

Three ideas this paper builds on

unified multimodal understanding and generationA single end-to-end model processes and generates multiple data types (e.g., images, text) with equal competence.28 papers →self-supervised learningA paradigm where models learn representations from unlabeled data by predicting hidden or transformed parts of the input.52 papers →vision-language modelA model that handles tasks combining visual and textual information, such as editing images from conversation or reasoning across images and text.234 papers →

Read it as

Models that both read images and make images have a lopsided skill set: they can often tell a good picture from a bad one but cannot draw to the same standard. Past fixes push in one direction only, using the judging skill to improve the drawing skill, or the reverse, and usually need human-checked examples or a separate scoring program to work.

This work lets the model train on its own multi-step text-and-picture outputs, such as a cooking tutorial. Each output is split into a text part and a picture part, and the system judges which part came out more trustworthy. The dependable part is then used as the target for the weaker one, so the teaching direction is chosen fresh for each example rather than fixed in advance.

Tested on two unified multimodal models, both skills rose: on one, a standard image generation benchmark went from 85.8 to 89.2 and a question-answering benchmark from 2282.6 to 2311.6. Self-training without an outside referee makes that coordination cheap to repeat.

How it works

before and after

RSS

The Daily Edition

The front page in your inbox each weekday morning: the lead, the picks, and why each one earned its place.

Concepts

unified multimodal understanding and generation self-supervised learning vision-language model rollout verification reinforcement learning with verifiable rewards knowledge distillation