Training multimodal AI models, which handle videos and images alongside text, is slow because different data samples take wildly different times to prepare. One short video might decode in milliseconds, while a long one takes seconds. In standard training, all computers wait for the slowest one to finish preparing its data before the next step, leaving fast computers idle and wasting expensive GPUs.
DataHelix changes this by treating the time each GPU spends processing a batch as a deadline for preparing the next batch. It then schedules data preparation across all computers to meet these deadlines, rather than just preparing data as fast as possible. It makes decisions about where to process data before decoding it, when data is small, avoiding costly transfers of large decoded files.
In tests with 11 different workloads on 8 accelerators, DataHelix reduced the rate at which data misses its deadline by 67.8% and cut the time GPUs wait for input by 87.0%. This means faster training without buying more hardware, directly cutting costs for teams working on video understanding and similar models.