Before a language model answers a long prompt, it must first process every token through all its layers. This prefill phase can take over two minutes on a single GPU for a 120,000-token input, making users wait before seeing any response.
RidgeFill avoids computing most of the model's upper layers for the prompt. It runs only the lower layers exactly up to a chosen split layer. From that split layer's state, it uses a simple regression model, fitted once from a single pass over calibration text, to predict each later layer's input. Each upper layer then writes its cache from that prediction and skips the rest of its computation for most prompt rows.
The method speeds time to first token by 1.25 to 1.60 times on different models and hardware, with a few points drop on retrieval benchmarks. No training is needed, and no tokens are dropped; only a subset of rows are computed exactly where errors would hurt most.