ConvMem is a training-free framework that reformulates long-context reasoning as hierarchical convolution. It treats an LLM as a convolutional kernel that summarizes text segments hierarchically, shortening the reasoning path from linear to logarithmic depth. The framework uses configurable strides and skip connections for robust evidence capture and detail preservation, and multi-kernel convolution to decompose complex queries into separate semantic channels. This enables massive parallelization across text segments and reasoning threads, reducing latency and error accumulation. On RULER-HotpotQA and RULER-2WikiMultiHopQA, ConvMem outperformed training-free baselines and showed superior generalization compared to RL-trained models, avoiding overfitting to dataset-specific patterns.
Hierarchical memory trees improve long-context reasoning without model training
ConvMem processes document segments in parallel and merges query-focused summaries through a logarithmic hierarchy rather than a sequential memory chain.
Research Lab
Hongming Zhang · Zhaozhen Gu · Fengshuo Bai · Ming Hao · Qingyang Zhang · Yuanyuan Wang · +3 more
Institute of Automation, Chinese Academy of Sciences
Research Digest··2 min read
In one line
ConvMem uses hierarchical convolution with strides, skip connections, and multi-kernel decomposition for parallel, training-free long-context reasoning.
Before you read this
Three ideas this paper builds oncontext compactionCompressing or summarizing an LLM's past context into a smaller representation to extend memory and improve reasoning within limited token budgets.71 papers →agent memoryA repository where an agent stores past interactions and outcomes to inform future decisions.200 papers →reinforcement learningA machine learning method where an agent learns to make sequences of decisions by receiving rewards or penalties for its actions.239 papers →
Read it as
Zhang et al. introduce ConvMem, a training-free framework that uses prompted language models as query-dependent summarization kernels over segments of long documents.
The authors recast long-context reasoning as hierarchical convolution: an LLM summarizes text segments with respect to a query, then repeatedly combines those summaries in a tree.
The Daily Edition
The front page in your inbox each weekday morning: the lead, the picks, and why each one earned its place.