The paper proposes SAGE, a framework that uses a (VLM) as an imperfect teacher to train a lightweight policy. During training, the agent queries the VLM only when its own policy entropy is high (indicating uncertainty), executes the VLM's suggested action, and distills that guidance into the policy using advantage-weighted behavioral cloning. This reduces VLM usage and avoids relying on the teacher at deployment.
Evaluated on sparse-reward visual reasoning and navigation tasks, SAGE improves over unguided RL in several environments, sometimes exceeding the VLM teacher itself. The approach is most beneficial when the VLM helps discover high-reward trajectories and less useful when unguided exploration already works or teacher actions are uninformative. It demonstrates that VLMs can serve as temporary, imperfect guides rather than fixed policies.