What they did

The authors developed Full-Group Policy Optimization (FGPO) for settings where a policy selects tools for a frozen reasoning model and every possible tool subset can be enumerated. Rather than estimate expected reward from a small group of sampled subsets, FGPO scores the full action space during each policy update.

They also precomputed rewards for every question–tool-subset pair, creating an exhaustive lookup table that removes frozen-reasoner inference from the training loop. FGPO and GRPO were compared across five frozen reasoners and three genomic reasoning benchmarks.

Key findings

• FGPO outperformed GRPO in all 15 reasoner–benchmark combinations, with an average gain of 6.75 percentage points and a maximum gain of 14.20 points.
• Under GRPO, questions producing no learning signal rose from 0.2% with a uniform reference policy to 20.8% after training, as the policy increasingly resampled identical subsets and group-normalized advantages collapsed.
• A standard on-demand GRPO schedule would require 2.4 times as many frozen-reasoner reward evaluations as FGPO's exhaustive precomputation.
• On GenomeQA, FGPO reduced the mean number of tools invoked per question from 2.36 to 1.40.

Why it matters

The results challenge the default use of sampled policy optimization when an agent's action space is small enough to enumerate. In specialist domains with a limited tool catalog, exact expectations can provide denser training signals, avoid duplicate-sampling pathologies, and produce more selective tool-use policies.

Caveats

FGPO depends on the complete tool-subset space being tractable and on rewards being precomputable for a frozen reasoner. Its storage and upfront evaluation costs may become prohibitive with larger, dynamic tool catalogs, while the reported evidence is confined to genomic benchmarks and does not establish similar gains in other scientific domains or with jointly trained reasoners.