Large language models can be customized with small “adapters” for specific tasks. LoRA is the most common adapter method, but others like DoRA and VeRA also exist. Serving systems batch requests for many LoRA adapters efficiently on one GPU, but reject other adapter types. Converting non-LoRA adapters to LoRA costs accuracy or memory.
The authors built Scute, a system that modifies the efficient LoRA kernel to handle five other adapter methods exactly, without conversion. They defined a general computation with six slots covering all methods and added efficient ways to compute the extra operations.
Scute matches the reference implementation with a mean error of at most 1.5% of each adapter's effect. Under a burst, a mix of methods takes only 5–17% more decode time than serving the same number of LoRA adapters. This allows platforms to serve multiple adapter methods together without accuracy loss or extra memory.