Tuesday 15 September 2026
Evening EditionRSS

ZOTPAPER

News without the noise

ZOTPAPER
RL for Agents

Paper-derived rubrics improve AI generation of scientific research plans

PaperGym separates research questions from method-and-experiment criteria, then uses those rubrics for self-teaching and reinforcement learning.

PaperBig Techcs.CLarXiv:2608.31119v1 ↗
Yuhan Wang · Zhengxi Lu · Yuchen Yan · Kaitao Song · Wenqi Zhang · Weiming Lu · +3 more

Zhejiang University · Apple

Research Digest··2 min read
Read the paper →
Thread:Context Engineering for Agents
The authors turn 20,000 scientific papers into training environments for research-plan generation, deriving prompts from each paper’s goals and background while extracting evaluation criteria from its methods and experiments. Across three Qwen3 model sizes, their rubric-centered training schedule produced higher benchmark scores than supervised fine-tuning, either training stage alone, or the same stages in reverse order.

What they did

The authors built PaperGym, a pipeline that converts papers into research-planning tasks with detailed grading rubrics. To reduce answer leakage, each question is synthesized from the paper’s research goal and background, while criteria covering methodological innovation and experimental design are derived separately from its methods and experiments.

They applied the rubric in two stages: first as privileged guidance for an OPSD self-teacher, then as the reward signal for group-relative policy optimization (GRPO). They trained Qwen3 models with 1.7B, 4B, and 8B parameters on the resulting 20,000-instance PaperGym-20k corpus and evaluated them on five benchmarks, including the released PaperGym-Innov and PaperGym-Design sets.

Key findings

  • PaperGym’s measured criterion leakage was 3.7%, compared with 11.90% to 34.10% in existing datasets.
  • The self-teaching-then-GRPO schedule improved five-benchmark averages by 5.6, 5.0, and 4.8 points for Qwen3-1.7B, 4B, and 8B, respectively, relative to the reported alternatives.
  • With the training recipe fixed, PaperGym-20k-trained models won 58.1% of three-way comparisons, versus 28.2% for models trained on RubricHub Science.
  • The trained Qwen3-8B scored 73.48 on ResearchQA, exceeding the authors’ reported result for the substantially larger Kimi K2.6 model.

Why it matters

Research plans usually lack a single verifiable answer, making reinforcement-learning rewards difficult to define. PaperGym shows that a paper’s internal structure can supply task-specific, multidimensional feedback while reducing the opportunity to earn rewards through paraphrase, offering a scalable route to training smaller models on open-ended scientific planning.

Caveats

The reported scores depend on rubrics automatically reconstructed from published papers, which may reward resemblance to known methods rather than genuinely novel or feasible research. Benchmark and model-judged gains do not establish that the generated plans lead to successful real-world experiments, and the abstract does not specify how well the pipeline transfers across scientific fields or handles flawed source papers.

§
newspaper

Research Digest

Articles published under the Zotpaper byline are synthesized from multiple source publications by our AI editor and reviewed by our editorial process. Each story combines reporting from credible outlets to give readers a balanced, comprehensive view.

Editorial ProcessSubscribe