SceneConductor breaks single-image 3D scene generation into three agent-based stages: initialization, environment construction, and refinement. In the first stage, object masks from the image are turned into coarse 3D representations and an initial layout. The second stage adds floors, walls, lighting, and materials using point-map geometry. A planner agent then fixes simple issues directly and sends complex ones to specialist agents that update only their local regions before reintegrating into the scene.
The key trick is a geometry-aware layout predictor trained on sparse geometric priors derived from point maps, avoiding expensive scene-level annotations. The authors report that this consistently outperforms prior methods on benchmarks for geometric accuracy, spatial consistency, and perceptual realism. It suggests that decomposing generation into specialized roles with grounded initialization can handle diverse real-world images better than holistic or weakly decomposed pipelines.