PaperGym turns each research paper into a training environment for research-plan generation. The key move is separating the two parts of the environment: the question is synthesized from the paper's research goal and background, while the grading rubric is derived from its method and experiments. This prevents the question from leaking the answer criteria, cutting criterion leakage to 3.7% compared with 11.90% to 34.10% in existing datasets.
Paper-derived rubrics improve AI generation of scientific research plans
PaperGym separates research questions from method-and-experiment criteria, then uses those rubrics for self-teaching and reinforcement learning.
Big Tech
Yuhan Wang · Zhengxi Lu · Yuchen Yan · Kaitao Song · Wenqi Zhang · Weiming Lu · +3 more
Zhejiang University · Apple
Research Digest··2 min read
In one line
Separating question sources from rubric criteria in paper-derived data cuts leakage to 3.7%; using the rubric as both teacher context and GRPO reward improves plan generation.
Before you read this
An idea this paper builds onRead it as
The authors turn 20,000 scientific papers into training environments for research-plan generation, deriving prompts from each paper’s goals and background while extracting evaluation criteria from its methods and experiments.
The authors built PaperGym, a pipeline that converts papers into research-planning tasks with detailed grading rubrics.
The Daily Edition
The front page in your inbox each weekday morning: the lead, the picks, and why each one earned its place.
What it released
Code