Large language models (LLMs) process input (prefill) and generate output (decode). Current energy estimates often ignore input length, but modern applications like agentic systems use long contexts, leading to underestimated energy use.
The authors systematically measured compute time and energy for various input and output lengths. They found that output time grows linearly with output length, but input time grows both linearly and quadratically with input length, meaning long inputs cost disproportionately more.
Input length significantly affects energy consumption, especially for large contexts. They also show that compute time can serve as a reliable proxy for energy when direct measurement is unavailable, enabling better estimation for proprietary models and supporting sustainable AI deployment.