The authors began from fractional differential equations, which naturally represent long-memory processes but are non-Markovian: computing the current state ordinarily requires access to the full history.
Fractional memory helps state space models retain longer context
FRAC approximates power-law memory with a finite bank of exponential modes, enabling efficient recurrence without the rapid forgetting typical of standard SSMs.
Chinese Tech
Ivan Kobyzev · Abbas Ghaddar · Ali Nasiri-Sarvi · Lifeng Shang · Yufei Cui
Huawei Noah’s Ark Lab, Montreal Research Center
Research Digest··2 min read
Kobyzev and colleagues introduce FRAC, a selective state space model built around fractional dynamics, whose influence from past inputs decays by a power law rather than exponentially.
Why this paper
From Huawei Noah’s Ark Lab, Montreal Research Center
In one line
FRAC, a selective SSM with fractional dynamics, outperforms other SSMs on long-context tasks.
What we could check
- ·No code link found
- ·No weights link found
- ·No dataset link found
- ·No compute details found
- ·No stated limitations found
- ·No benchmark numbers found
Observed from the paper text and links we have. Absence here means we did not find it, not that it does not exist.
§