The paper introduces Hermes, a family of configurable harnesses that give language models varying degrees of control over how to allocate new context windows and what information to carry between them during reasoning. It also presents Hermes-Learn, a two-stage training framework (supervised fine-tuning followed by reinforcement learning) to teach models this contextual reasoning capability.
Capable models can exploit the flexibility of Hermes to scale their performance with additional inference-time compute, but smaller open-source models initially perform worse than when reasoning in a single context. Training with Hermes-Learn closes this gap, producing models that adapt their contextual reasoning strategies based on the problem and the current state of reasoning.
The gains from this learned contextual reasoning generalize across multiple benchmarks and model families, extrapolate to inference-time compute budgets beyond those seen during training, and transfer to other methods beyond the Hermes harnesses themselves.