Most video quality metrics focus on aesthetics or text-video alignment, so subtle object motion failures go unnoticed. The authors built VidMotion, a diagnostic dataset of 6,879 videos, including 5,713 generated videos from nine models and 1,166 real videos, annotated with scores for object consistency, motion continuity, and physical plausibility plus failure causes. They also propose MotionInsight, an evaluator that uses motion-aware representations from tracking and camera motion instead of relying only on sampled RGB frames.
MotionInsight aligns its scoring with human judgments using motion description alignment and GRPO with motion-specific rewards. Experiments show it produces human-aligned scores on the three dimensions and grounded explanations of concrete failures. This matters because it shifts video evaluation toward diagnosing why motion is wrong, not just rating overall quality.