What they did

The authors trained lightweight reinforcement-learning agents on sparse-reward visual reasoning and navigation tasks. During training, SAGE measures the learner’s action uncertainty using policy entropy and queries a vision-language model only when that uncertainty is high; the teacher’s suggested action is then executed to generate experience.

SAGE distills these actions into the learner’s policy. To limit imitation of bad advice, the framework can weight each teacher action by its environment-derived advantage—an estimate of whether that action produced better outcomes than the learner’s baseline expectation.

Key findings

• SAGE improved over unguided reinforcement learning in several tested environments, particularly where VLM guidance helped uncover otherwise difficult high-reward trajectories.
• In some settings, the autonomous learned policy exceeded the performance of the VLM teacher, indicating that interaction-based learning could correct or move beyond imperfect demonstrations.
• Selective querying reduced teacher use by limiting VLM calls to a fraction of training steps rather than consulting it continuously.
• The trained policy required no VLM calls at evaluation time, shifting the teacher’s computational cost entirely to training.

Why it matters

The results support using large vision-language models as temporary exploration guides rather than fixed decision-makers. This can transfer their prior knowledge into cheaper policies while allowing environmental feedback to determine which advice should be retained, potentially reducing inference costs and repeated teacher errors.

Caveats

The abstract does not report task counts, absolute performance gains, query rates, or computational costs, so the scale and consistency of the improvements cannot be assessed here. The method also helps less when ordinary exploration already finds rewards or when teacher actions fail to generate informative experience, making its value dependent on both the environment and teacher quality.