The authors measured variation in prefill and decode demand using a large internal LLM fleet and a public agentic workload trace.
Elastic phase sharing improves efficiency in agentic LLM serving
Crossflow lets decode nodes temporarily absorb prefill work through bounded leases, reducing idle capacity without slow replica reassignment.
Industry
Yi Xu · Ehsan K. Ardestani · Wenyin Fu · Martin Schatz · Krishna Malladi · Zhan Shu · +4 more
Meta Platforms
Research Digest··2 min read
Xu et al.
Why this paper
From Meta Platforms
In one line
Crossflow makes the prefill-decode boundary elastic by letting decode nodes lend spare capacity to prefill, improving token throughput 16.2-17.4% over static P/D disaggregation.
What we could check
- ·No code link found
- ·No weights link found
- ·No dataset link found
- ✓Compute or model size stated (hardware NVIDIA GB300 servers)
- ·No stated limitations found
- ✓Reports numbers on named benchmarks
Observed from the paper text and links we have. Absence here means we did not find it, not that it does not exist.
§