Merging reasoning models layer by layer cuts answers by up to 93%

A training-free method that decides how much of each layer to blend using an estimate of the loss curvature beats existing merging baselines on six maths benchmarks, gaining over 4% accuracy while sharply shortening outputs.

Chinese Tech

Zeqiu Yu · Xuesheng Zhang · Wenxiao Zhao · Shixiao Wang

University of Virginia · University of California, Los Angeles · University of Birmingham · Baidu, Inc.

Research Digest·4 November 2026·3 min read

Illustration: Merging reasoning models layer by layer cuts answers by up to 93%

In one line

Low-Rank Hierarchical Merging outperforms prior methods on long-to-short reasoning model merging by using second-order curvature and dual constraints.

Read it as

Long reasoning models that show step-by-step thinking (chain-of-thought) are accurate but long-winded, making them costly to deploy. Short answers save resources but often hurt accuracy.

This paper introduces a training-free way to merge the long model with a compact base model. It measures each layer's “curvature” (how quickly the error changes when mixing parameters) and uses that to set per-layer merge strengths, while also enforcing a target for final output length.

On math benchmarks, the method improves accuracy by 4.2% over prior merging techniques and cuts output length by 93.1% relative to the long model. This enables deploying strong reasoning models efficiently without retraining.

How it works

before and after

RSS

The Daily Edition

The front page in your inbox each weekday morning: the lead, the picks, and why each one earned its place.

Concepts

model merging long-to-short reasoning second-order curvature low-rank hessian approximation chain of thought