What they did
The authors built a data pipeline that identifies retrieval-relevant events and entities in diverse videos, gathers supporting information through video grounding and Web retrieval, and turns that evidence into question-answer pairs with task-specific tool-use trajectories.
They then filtered tool calls according to the target model’s existing video-understanding and internal-knowledge capabilities. AdaVDR was trained through supervised fine-tuning followed by reinforcement learning using a redundancy-aware reward, encouraging necessary tool use and selective reflection when intermediate results appeared unreliable.
Key findings
- AdaVDR achieved the strongest reported performance among the open-source models evaluated on VDR-EE, the authors’ new benchmark of entity-centric and event-centric video questions.
- The method substantially improved its base models on the existing VideoDR benchmark, indicating that the gains were not confined to the newly constructed evaluation set.
- Model-conditioned filtering removed tool calls or entire tool chains that the target model could bypass, tailoring training trajectories to its existing capabilities.
- The learned agent invoked tools according to the question and video type and used backtracking selectively rather than applying a fixed retrieval-and-reflection sequence.
Why it matters
Video research agents must decide not only how to interpret footage, but also when external evidence is needed. AdaVDR suggests that capability-aware tool selection can reduce avoidable retrieval steps and their associated latency and error risk while preserving access to Web knowledge for questions that require it.
Caveats
The provided results do not include numerical scores, latency measurements, tool-call reductions, or statistical uncertainty, making the size and efficiency of the improvement difficult to assess. VDR-EE was created with the same pipeline used to train the approach, so evaluation on independently constructed datasets, changing Web content, and more varied videos remains important.