← All concepts

sparse attention

Attention normally lets each word look at every previous word, costing huge compute as text grows. Sparse attention instead picks only a few important tokens per word, cutting that cost. But deciding which tokens matter can itself be slow and can leak privacy, because the pattern of what gets skipped reveals information about the content. The key idea is to keep the benefits, speed, long-context handling, while avoiding the pitfalls: selecting important tokens cheaply, preserving quality, and protecting data.

Across the papers, the common thread is that sparse attention is no longer just a speed hack; it is being reworked as a principled, trained or training-free method that reduces computation without hurting output. Several approaches focus on better selection, using learned routers, community detection, or predictive indexers, while others target specific domains like video, where attention patterns repeat. All report large speedups and note trade-offs in accuracy or privacy, showing the technique is mature enough for real systems but still needs care.

Papers
51
Released code
4
First seen
Sept 2026
Latest
Nov 2026

50 papers in the last two months, against 1 in the two before.

Start here

Three papers to read first, in the order they came out
  1. 1Million-token prompts served at 2.3 times the throughput without retraining

    Retrofitting pretrained dense-attention large language models with chunk-level sparse attention, without retraining, speeds long-context serving while keeping benchmark scores close to dense attention.

    Nov 2026
  2. 2Overlapping indexer scan with attention speeds up long-context decoding by up to 14%

    A sparse-attention layer can attend the token selection its previous layer's indexer already made, letting the scan overlap attention and speeding up long-context decoding.

    Nov 2026
  3. 3Sparse attention for long-context AI verifies its own quality in one pass

    Coverage-Verified Sparse Attention achieves closed-loop quality control by using the softmax mass fraction to verify and correct sparse attention outputs.

    Nov 2026

Who is working on it

NVIDIA 4Alibaba Group 4National University of Singapore 3Amazon AGI 2Baidu Inc. 2The Hong Kong University of Science and Technology 2

The papers

Most central to this idea first, not most recent.

Concepts are extracted from each paper and reused across the corpus, so this page grows on its own as the desk reads.