The authors derive a regularized leave-one-out objective that separates attention-based reading from state-dependent scaling, leading to the RefineICL architecture: an attention-gated contextual stack without feedforward networks, with selected low-rank feature interaction and typed memory.
Tabular foundation models refine representations in-context with attention-gated updates
The authors introduce in-situ representation refinement and the RefineICL model, which outperforms TabPFN-3 while using no feedforward layers.
Chinese Tech
Tian Zhou · Beverly Jin · Linxiao Yang · Xue Wang · Wenwei Wang · Bingqing Peng · +3 more
Ant Group · Independent Researcher
Research Digest··2 min read
Thread:Tabular Foundation Models
The authors develop in-situ representation refinement, where support labels guide changes to the episode's representations during a forward pass.
Why this paper
From Ant Group and Independent Researcher
In one line
Tabular foundation models refine in-context representations via attention-gated updates, improving accuracy and memory efficiency over TabPFN.
What we could check
- ·No code link found
- ·No weights link found
- ·No dataset link found
- ·No compute details found
- ·No stated limitations found
- ✓Reports numbers on named benchmarks (2 benchmarks)
Observed from the paper text and links we have. Absence here means we did not find it, not that it does not exist.
§