Current multimodal large language models encode an entire table image once, making it difficult to focus on specific rows, columns, or cells as reasoning steps progress. InterTab addresses this by interleaving chain-of-thought reasoning with tool calls that crop structure-aligned table regions—such as a particular row or cell—so each reasoning step can inspect the relevant visual evidence. The method is trained in two stages: supervised structure-aware alignment on a new dataset (InterTab-22K) of step-by-step trajectories, followed by active localization optimization that rewards answer correctness, precise bounding boxes, and proper tool usage. On nine table reasoning benchmarks, InterTab raises the average accuracy of its backbone from 68.28% to 73.17%, achieving the best overall performance among compared methods.
Interleaved table crops improve reasoning over dense visual layouts
InterTab lets a multimodal model request row, column and cell crops while reasoning, linking each step to specific table evidence.
Chinese Tech
Hanqian Li · Sirui Huang · Chen Ling · Jungang Li · Yu Huang · Kening Zheng · +9 more
The Hong Kong University of Science and Technology (Guangzhou) · Zhejiang University · The Hong Kong University of Science and Technology · City University of Hong Kong · University of Illinois at Chicago
Research Digest··2 min read
In one line
InterTab interleaves chain-of-thought with structure-aligned region crops to improve multimodal table reasoning.
Before you read this
Three ideas this paper builds onstructured layout reasoningA multi-step process that generates or verifies spatial arrangements of objects against explicit constraints and geometry.18 papers →vision-language modelA model that handles tasks combining visual and textual information, such as editing images from conversation or reasoning across images and text.265 papers →group relative policy optimizationA reinforcement learning method that optimizes policy by computing advantages based on group-level reward statistics.188 papers →
Read it as
The authors train a multimodal language model to alternate between chain-of-thought reasoning and tool calls that crop relevant regions from a table image.
The authors introduce InterTab, a framework for actively acquiring visual evidence during table reasoning.
How it works
before and afterThe Daily Edition
The front page in your inbox each weekday morning: the lead, the picks, and why each one earned its place.