Google open-sources ML Drift, a cross-platform GPU engine for on-device AI

Engine powers YouTube Shorts, Photos and Snapchat effects with latency reductions of up to 40 per cent

By LineZotpaper AI staff

Published25 October 2026

Read Time3 min

Illustration: Google open-sources ML Drift, a cross-platform GPU engine for on-device AI

Google has released ML Drift, an open-source, cross-platform GPU compute engine designed for on-device AI and machine learning inference. Licensed under Apache 2.0, the engine already powers applications including Chrome, YouTube Shorts, Google Photos and Meet on millions of devices.

Google announced the release of ML Drift, an open-source GPU compute engine built to simplify the development of on-device AI and machine learning across fragmented hardware environments. The engine abstracts different GPU architectures and low-level APIs, supporting OpenGL ES, OpenCL, Metal and WebGPU. It serves as the core GPU acceleration engine for LiteRT, the successor to TensorFlow Lite.

The company says ML Drift delivers “stability for traditional ML and peak performance for the most advanced generative AI”. It introduces technical improvements over the legacy TFLite GPU delegate, including unified shaders through tensor virtualization, extensible custom operations and five-dimensional tensor support. ML Drift incorporates hardware-specific optimisations for GPUs from Arm, Intel and Qualcomm Technologies. For desktop inference, it supports Windows and Linux through Dawn, Chromium's WebGPU implementation, while retaining native Metal backend support on macOS.

Google benchmarked ML Drift using Gemma models on developer-grade workstation hardware and reported 12 per cent lower memory overhead compared to other frameworks including llama.cpp and MLX LM. The company highlighted that ML Drift is already in production across its own products and partners. In YouTube Shorts, it achieves up to a 40 per cent reduction in average frame latency for segmentation-based effects on Android and iOS. Google Photos sees speedups of up to two seconds for computational photography and segmentation pipelines. Adobe Lightroom and Photoshop benefit from performance improvements of up to 30 per cent for AI-powered features such as Select Subject and Select Sky, while Snapchat reports a 30 per cent reduction in model latency for ML-powered face and style-generation effects on Android.

Commenting on the announcement on LinkedIn, AI/ML engineer Pavithra Agraharam argued that “native 5D tensor support without layout hacks is a massive win for running spatial and transformer models on edge GPUs”, and that decoupling logical tensors from physical memory should “significantly cut down on memory fragmentation during live inference”. Matthew Charles Vladislav Busel, founder and sole developer of Tensorust, questioned “how it compares with Core ML on recent iPhones for latency and memory, since that is what most of us ship against today”, a comparison Google has not yet provided.

With the release of ML Drift, Google will stop updating the legacy TensorFlow Lite GPU delegate. The company recommends that developers migrate to the LiteRT ML Drift GPU accelerator for backward compatibility with existing models. GPU and system engineers can also access the engine's lower-level C++ API to construct custom GpuModel compute graphs and write custom shaders.

RSS

The Daily Edition

The front page in your inbox each weekday morning: the lead, the picks, and why each one earned its place.

Analysis

Why This Matters

Standardises on-device AI development: ML Drift simplifies the highly fragmented landscape of GPU architectures and low-level APIs, potentially accelerating the deployment of advanced AI on edge devices.
Proven at scale: The engine is already running on millions of devices through Google's own apps and external partners such as Adobe and Snap, giving it immediate credibility and a vast real-world testing base.
Sets a new technical direction: By open-sourcing ML Drift and discontinuing the legacy TFLite GPU delegate, Google is establishing a new baseline for on-device inference that could influence the broader mobile and edge AI ecosystem.

Background

Developers building on-device AI applications have long faced a fragmented landscape of GPU architectures, driver versions and low-level APIs covering OpenGL, OpenCL, Metal and WebGPU. Google's previous offering, the TensorFlow Lite GPU delegate, was the primary tool but required significant hardware-specific tuning. ML Drift is Google's next-generation response, acting as the core GPU acceleration engine for LiteRT, the successor to TensorFlow Lite, and is designed to provide a unified foundation for portable and performant on-device inference.

Key Perspectives

Pavithra Agraharam, AI/ML engineer: Said native 5D tensor support “without layout hacks is a massive win for running spatial and transformer models on edge GPUs” and that decoupling logical tensors from physical memory should “significantly cut down on memory fragmentation during live inference”. Matthew Charles Vladislav Busel, founder and sole developer of Tensorust: Questioned how ML Drift compares with Core ML on recent iPhones for latency and memory, noting that is what most developers “ship against today”, a comparison Google has not yet provided.

What to Watch

Independent benchmarks: Whether Google or third parties provide the performance comparisons against platforms such as Apple's Core ML that developers have requested.
Developer adoption: How quickly the broader developer community moves from the legacy TFLite delegate to the ML Drift accelerator within LiteRT.
Hardware partner expansion: Whether hardware-specific optimisations extend beyond the initial Arm, Intel and Qualcomm support to include other GPU vendors such as AMD and Apple Silicon.

Sources

Google ML Drift Accelerates On-Device AI Inference Across Diverse GPUs · InfoQ

Checked

12 claims in this story checked against 1 source: 10 supported.

Show each claim

SupportedGoogle has released ML Drift, an open-source, cross-platform GPU engine for on-device AI.InfoQ: “Google has released ML Drift, an open-source, cross-platform GPU compute engine designed for on-device AI and machine learning inference.”
Partly supportedThe engine powers YouTube Shorts, Photos and Snapchat effects with latency reductions of up to 40 per cent.InfoQ: “In YouTube Shorts, ML Drift achieves up to a 40% reduction in average frame latency for segmentation-based effects on Android and iOS. In Google Photos, it speeds up computational photography and segmentation pipelines by up to two second. ... Snapchat sees a 30% reduction in model latency”
SupportedML Drift is already running on millions of devices through Chrome, YouTube Shorts, Google Photos and Meet.InfoQ: “The company also emphasized that ML Drift is already running on millions of devices through apps like Chrome, YouTube Shorts, Photos, and Meet.”
Not in the sourcesLiteRT is the successor to TensorFlow Lite.InfoQ: “ML Drift is the core GPU acceleration engine that powers LiteRT.”
SupportedGoogle says ML Drift delivers “stability for traditional ML and peak performance for the most advanced generative AI”.InfoQ: “Google says, delivering "stability for traditional ML and peak performance for the most advanced generative AI".”
SupportedGoogle's benchmark found 12 per cent lower memory overhead than other frameworks, including llama.cpp and MLX LM.InfoQ: “Google benchmarked ML Drift using Gemma models on developer-grade workstation hardware, reporting a 12% lower memory overhead compared to other frameworks, including llama.cpp and MLX LM.”
SupportedGoogle Photos pipelines speed up by up to two seconds.InfoQ: “In Google Photos, it speeds up computational photography and segmentation pipelines by up to two second.”
SupportedAdobe Lightroom and Photoshop get performance improvements of up to 30 per cent for AI-powered features such as Select Subject and Select Sky.InfoQ: “Adobe Lightroom and Photoshop benefit from performance improvements of up to 30% for selected AI-powered photo-editing features such as Select Subject, Select Sky, and Adaptive Portrait.”
SupportedSnapchat reports a 30 per cent reduction in model latency for ML-powered face and style-generation effects on Android.InfoQ: “Finally, Snapchat sees a 30% reduction in model latency for ML-powered face and style-generation effects on Android, enabling the use of larger diffusion models.”
SupportedPavithra Agraharam said native 5D tensor support without layout hacks is a massive win for running spatial and transformer models on edge GPUs.InfoQ: “Native 5D tensor support without layout hacks is a massive win for running spatial and transformer models on edge GPUs.”
SupportedMatthew Charles Vladislav Busel questioned how ML Drift compares with Core ML on recent iPhones for latency and memory.InfoQ: “Matthew Charles Vladislav Busel, founder and sole developer of Tensorust, questioned " how it compares with Core ML on recent iPhones for latency and memory, since that is what most of us ship against today"”
SupportedGoogle will stop updating the legacy TensorFlow Lite GPU delegate.InfoQ: “With the release of ML Drift, Google will stop updating the legacy TensorFlow Lite GPU delegate.”