6-35B-A3B with supervised examples from a diverse synthetic curriculum.
Small agents learn to navigate documents far beyond their context
A fine-tuned model with an 8,000-token window matched a million-token frontier model on one synthetic long-document benchmark.
Industry
Bryce Sandlund
Research Digest··3 min read
6-35B-A3B to process long documents through two tools: one reads selected token ranges, while the other delegates subtasks to fresh copies of the model.
Why this paper
Industry
In one line
A small model with 8K tokens of context matches GPT-5.4 with 1M tokens on long-context benchmarks beyond 40K tokens.
What we could check
- ·No code link found
- ·No weights link found
- ·No dataset link found
- ·No compute details found
- ✓Limitations stated by the authors (2 noted)
- ✓Reports numbers on named benchmarks
Observed from the paper text and links we have. Absence here means we did not find it, not that it does not exist.
§