Browser-game training makes AI webpage generation surpass a frontier model

Trained on under 1,000 requests, mostly browser mini-games, the authors' framework improves a 35-billion-parameter model's scores on game and standard webpage benchmarks.

Big Tech

Xujia Wang · Yixuan Liu · Shaohan Huang · Furu Wei · Lei Hou · Bin Xu · +1 more

Tsinghua University · Microsoft Research

Research Digest·4 November 2026·2 min read

Illustration: Browser-game training makes AI webpage generation surpass a frontier model

In one line

WebCatalyst uses high-fidelity reinforcement learning to significantly improve interactive HTML generation, outperforming Claude Opus 4.7.

Before you read this

Three ideas this paper builds on

reinforcement learningA machine learning method where an agent learns to make sequences of decisions by receiving rewards or penalties for its actions.259 papers →credit assignmentDetermining which actions or steps in a sequence contributed to a final outcome, often for reinforcement learning.180 papers →reward hackingReward hacking occurs when an AI agent exploits loopholes to achieve high reward without meeting the intended objective.58 papers →

Read it as

Current AI models can generate HTML webpages from text descriptions, but the pages often look good yet break during actual use, especially for complex interactive tasks like games. Better training feedback is needed.

WebCatalyst introduces three improvements to the way the model learns from trial and error () for HTML generation. It uses a system that actually plays the game to evaluate the page, a visual check to stop the look from getting worse, and a way to trace which specific parts of the code caused successful actions.

With fewer than one thousand training requests, WebCatalyst raised a mini-game benchmark score from 68.8 to 75.3 and also improved conventional webpage generation, surpassing a strong baseline. This shows that high-fidelity feedback is crucial for generating truly interactive and usable HTML.

How it works

stage by stage

RSS

The Daily Edition

The front page in your inbox each weekday morning: the lead, the picks, and why each one earned its place.

Concepts

reinforcement learning credit assignment reward hacking large language model vision-language model