The authors designed HyCo for the mobile-cloud setting, where a server LLM verifies draft tokens from a device.
Hybrid drafters split depth and breadth to cut mobile-cloud GPU costs
HyCo assigns the verifier's native multi-token prediction head to supply depth and one SLM pass for alternatives, reducing server GPU time per token by up to 38.6%.
Big Tech
Hanbin Bae · Heejin Choi · Joohyung Lee · Byoung Jin Choi · Sangjun Park · Hoon-Young Cho
Samsung Research
Research Digest··2 min read
Bae et al.
Why this paper
From Samsung Research
In one line
HyCo reduces server GPU time by 38.6% using a hybrid drafter with MTP head and SLM on mobile.
What we could check
- ·No code link found
- ·No weights link found
- ·No dataset link found
- ✓Compute or model size stated (hardware RTX PRO 6000)
- ·No stated limitations found
- ✓Reports numbers on named benchmarks (2 benchmarks)
Observed from the paper text and links we have. Absence here means we did not find it, not that it does not exist.
§