TrackTok learns to group video features into semantic regions, usually objects, without object annotations for that grouping.
Object-aligned video tokens preserve generation quality with fewer latent units
Pham and colleagues report comparable UCF-101 generation quality using eight times fewer tokens, alongside stronger low-token-budget frame prediction on Kinetics-600.
Research Lab
Nhi Pham · Christopher Wewer · Bernt Schiele · Jonas Fischer · Jan Eric Lenssen
Max Planck Institute for Informatics · Saarland University
Research Digest··2 min read
Pham and colleagues introduce TrackTok, a video tokenizer that compresses footage into representations organized around persistent objects rather than fixed image locations.
The Daily Edition
The front page in your inbox each weekday morning: the lead, the picks, and why each one earned its place.