structured layout reasoning
Structured layout reasoning is about explicitly planning the arrangement of a task before executing it. For instance, a table reader might first locate a specific row or cell and then inspect only that region. An image generator might decide where each person stands before drawing faces. A 3D scene builder might separate geometry, materials, and lighting into stages, using a layout to guide each step. The structure itself is reasoned about, not left implicit.
The papers commonly find that such explicit planning improves control and accuracy. Systems that decompose tasks, whether by cropping regions, separating rules from rendering, or predicting layouts before generation, outperform holistic ones. They also reduce errors like identity mixing or missing pieces and often need fewer resources. Deliberate layout reasoning emerges as a powerful strategy for complex multimodal and interactive tasks.
- Papers
- 17
- Released code
- 1
- First seen
- May 2026
- Latest
- Nov 2026
11 papers in the last two months, against 3 in the two before.
Start here
Three papers to read first, in the order they came out- 1Video generators follow user-defined rules better when separate software controls object behavior
BoxWorld separates interaction reasoning from generative rendering to create real-time interactive video worlds with user-defined behaviors.
Oct 2026 - 2Multi-agent framework learns to coordinate puzzle piece assembly without planning
Multi-agent reinforcement learning with a graph neural network can coordinate multiple parts to assemble Tangram puzzles simultaneously.
Nov 2026 - 3Driving simulators follow planned paths more faithfully with overhead views
BirdDreams jointly generates perspective and bird's-eye-view videos to create reactive driving simulations from a single image.
Nov 2026
How fast it is moving
Often studied with
vision-language modelA model that handles tasks combining visual and textual information, such as editing images from conversation or reasoning across images and text.5 togetherfine-grained spatial reasoningReasoning precisely about object positions, orientations, and spatial relations across planar, depth, and temporal dimensions for robotics and navigation.4 togetherworld model for agentsA predictive environment model that agents use to simulate outcomes, enabling planning, reasoning, and long-horizon task execution.3 togetherThe papers
Most central to this idea first, not most recent.
- Big Techcs.CV
Driving simulators follow planned paths more faithfully with overhead views
BirdDreams jointly generates perspective and bird's-eye-view videos to create reactive driving simulations from a single image.
NVIDIA, KAIST · Nov 2026
- Academiccs.RO
Multi-agent framework learns to coordinate puzzle piece assembly without planning
Multi-agent reinforcement learning with a graph neural network can coordinate multiple parts to assemble Tangram puzzles simultaneously.
Technical University of Darmstadt · Nov 2026
- Chinese Techcs.CV
Video generators follow user-defined rules better when separate software controls object behavior
BoxWorld separates interaction reasoning from generative rendering to create real-time interactive video worlds with user-defined behaviors.
University of British Columbia, Vector Institute for AI · Oct 2026
- Chinese Techcs.CV
Sparse aerial photos become persistent, agent-editable 3D city models
Sparse aerial observations can be converted into persistent, agent-operable 3D cities with metric geometry, road connectivity, and editable entity identities.
Nanjing University, Shanghai Artificial Intelligence Laboratory · Oct 2026
- Chinese Techcs.CV
Interleaved table crops improve reasoning over dense visual layouts
InterTab interleaves chain-of-thought with structure-aligned region crops to improve multimodal table reasoning.
The Hong Kong University of Science and Technology (Guangzhou), Zhejiang University · Oct 2026
- Top Universitycs.AI
Recursive latent reasoner boosts diffusion models for visual puzzles
A recursive reasoner coupled to a frozen diffusion model solves visual reasoning puzzles from pixels without symbolic supervision.
Warsaw University of Technology, IDEAS Research Institute · Oct 2026
- Academic
Web agents struggle to turn search results into usable artifacts
Frontier computer-use agents fully complete fewer than 3% of long-horizon web tasks requiring synthesis and artifact creation, and visual/spatial failures often make partial artifacts unusable.
University of Utah · Sept 2026
- Top Universitycs.IRcode
VikingRAG Cuts Token Use for Retrieval Over Structured Documents
VikingRAG matches state-of-the-art RAG accuracy while consuming 11.6% to 51.9% of prior tokens, dropping to 5.1% to 32.5% with trace reuse and adaptive escalation.
Renmin University of China, Independent Researcher · Sept 2026
- Chinese Techcs.CV
Explicit identity-layout planning improves group images with specified people
Explicit identity-layout grounding enables identity-preserving generation to scale to larger groups without direct reference-face copying.
Fudan University, Tencent · Aug 2026
- Top Universitycs.CV
Multi-agent framework iteratively verifies interior layouts against architectural constraints
Agentic Designer uses three agents to iteratively generate interior layouts that adhere to architectural constraints.
Guangdong University of Technology, Sun Yat-sen University · July 2026
- AI Startupcs.CV
Multi-agent orchestration generates 3D scenes from single images
SceneConductor generates coherent 3D scenes from single images via three-stage multi-agent orchestration and geometry-aware layout prediction.
Nanyang Technological University, University of Oxford · June 2026
- Independentcs.CV
Code-driven agents generate images via sketching and texturing stages
GenClaw stages image generation into conceptualize, sketch (via code), and color, enabling direct canvas manipulation and interpretability.
May 2026
- Chinese Techcs.CV
Visual skill representations reduce exposure-driven action bias in language model agents
Language models increasingly favor an action when its procedural pattern is repeated in a skill; visual skill representation reduces that bias.
Northeastern University, Alibaba Group · Oct 2026
- Independentcs.CL
Document parser reads hundreds of pages without growing per-step compute
Unlimited OCR 2 achieves constant computational cost across arbitrarily long documents, enabling one-shot hundred-page parsing.
Oct 2026
- Big Techcs.CV
Autoregressive model generates road and building vectors in one pipeline
Treating vectorized map elements as a formal language lets one auto-regressive model generate roads and buildings end to end, with no post-processing.
Microsoft · Oct 2026
- Chinese Techcs.CV
Benchmark exposes image generators’ limits on dense bilingual visual text
UltraText Bench exposes distinct fidelity, clarity, spatial, and scene-quality failures in dense English and Chinese text generation as workloads increase.
Westlake University, Ant Group · Oct 2026
- Top Universitycs.AI
Multi-agent system creates editable 4D scenes from text prompts
SimWorlds generates dynamic 3D scenes from text using multi-agent planning and verification, outperforming prior baselines.
Carnegie Mellon University, Harvard University · July 2026
Concepts are extracted from each paper and reused across the corpus, so this page grows on its own as the desk reads.