CLAMP adds decoding-time constraints to a frozen vision-language planner. It uses the initial observation to restrict object references to what is visible, and a symbolic action model to mask next-token candidates that violate action syntax, preconditions, or goals. An HMM lookahead then reweights feasible candidates by goal reachability. This keeps the VLM's language prior while blocking visually unsupported or unsafe actions. Experiments on VLABench, SafeAgentBench, and TaPA show improved object grounding and safety, with remaining failures mostly attributed to perception errors or misaligned constraint specifications.
Scene-grounded decoding keeps vision-language plans executable and visually supported
CLAMP combines hard token masks with probabilistic world-state lookahead to constrain plans from a frozen vision-language model.
Independent
Tianyi Ma · Parisa Kordjamshidi
Research Digest··2 min read
In one line
Scene-grounded constrained decoding with hard masks and HMM lookahead improves vision-language embodied plan grounding and safety.
Before you read this
Three ideas this paper builds onconstrained decodingA generation technique that imposes constraints like safety or validity to guide output, improving reliability.3 papers →vision-language modelA model that handles tasks combining visual and textual information, such as editing images from conversation or reasoning across images and text.216 papers →world model for agentsA predictive environment model that agents use to simulate outcomes, enabling planning, reasoning, and long-horizon task execution.96 papers →
Read it as
Ma and Kordjamshidi introduce CLAMP, a decoding framework that converts visual scene evidence and a symbolic action model into constraints on an embodied planner.
CLAMP uses an initial visual observation to limit object references to entities supported by the scene.
The Daily Edition
The front page in your inbox each weekday morning: the lead, the picks, and why each one earned its place.