Google, working with Meta, has open-sourced TorchTPU, a native PyTorch backend for Google's tensor processing units that lets existing PyTorch workloads run on TPU hardware with minor code changes, InfoQ reports.
The project supports eager and compiled execution, distributed computing and hardware-specific optimisations. According to InfoQ, moving a workload from GPUs to TPUs mainly means replacing the cuda device with tpu and the nccl distributed backend with tpu_dist, or using the new torch.accelerator.current_accelerator API to detect whichever accelerator is available.
Google began enabling PyTorch workloads on TPUs in 2020 through PyTorch/XLA, which relied on JAX, XLA and lazy tensors. According to Google, that approach required intrusive source-code changes to define compilable regions, suffered severe compile-time latency, did not support PyTorch's native eager execution mode and lagged in supporting emerging distributed computing primitives in the PyTorch ecosystem. TorchTPU is built around a different design, with eager execution as a priority.
It offers three eager execution modes: strict, debug and DeferAndFuse. The last is a hybrid eager and compiled mode that Google claims "can be 50% or more faster than Strict Eager while reducing peak HBM memory".
To test whether existing models run unmodified, Google built a pipeline to evaluate about 5,000 Hugging Face models across more than 500 architectures in both eager and compiled modes, on TPU hardware generations starting from v5e. InfoQ describes the selection as a two-tiered process intended to balance real-world adoption with architectural diversity and avoid testing near-duplicate variants.
Google also measured a 30-billion parameter Muse Glimmer model written in standard PyTorch. On a single TPUv7 chip, inference throughput reached 25.7k tokens per second at a sequence length of 5,120 and 22.4k at 10,240. For training on four TPUv7 chips, throughput reached 5.6k and 4.6k tokens per second at the same sequence lengths.
Separate experiments used the Qwen3 MoE 235B model for training workloads, scaling from 1,024 to 8,192 TPUv7 chips. For inference on large-scale mixture-of-experts workloads using vLLM, Google reported competitive performance per dollar.
On Hacker News, user tingkuanpei wrote: “Very interesting work. Google should have done this a long time ago. PyTorch's ecosystem is just so rich.” Other users pointed out that beyond offering TPUs through Google Cloud, Google sells them to enterprise clients for on-premises deployments, naming Anthropic among them and suggesting Meta could follow.
InfoQ reports that TorchTPU is still at an early stage and evolving. Its roadmap covers dynamic shapes, compilation performance, operator coverage, reinforcement learning and quantization.
Analysis
Why This Matters
- PyTorch is the dominant framework for much of the machine learning work done today, so a native, low-friction path onto Google's TPUs widens the hardware options available to teams that have so far been tied closely to Nvidia GPUs.
- Google's own account of PyTorch/XLA's shortcomings is an admission that its previous route onto TPUs was costly for developers; TorchTPU is an attempt to remove that migration tax.
- Because Meta co-developed the project, it signals interest from a major PyTorch steward in keeping the framework portable across accelerator types rather than tied to one vendor.
Background
Google designs TPUs as custom chips for machine learning work and sells access to them through its cloud, as well as selling hardware for on-premises deployments. PyTorch, originally developed at Meta, is one of the two most widely used open-source deep learning frameworks, alongside Google's own TensorFlow. Running PyTorch on non-Nvidia hardware has historically required translation layers, and Google's first attempt, PyTorch/XLA, was built on the XLA compiler and JAX.
Key Perspectives
Google: says PyTorch/XLA had significant shortcomings, including intrusive source-code changes, severe compile-time latency, no native eager execution and lagging distributed primitives, and says TorchTPU's DeferAndFuse mode can be 50% or more faster than strict eager execution while reducing peak HBM memory.
Hacker News user tingkuanpei: calls the work "very interesting" and says Google should have done it a long time ago because PyTorch's ecosystem is so rich.
Other Hacker News users: point out that Google also sells TPUs for on-premises enterprise deployments, naming Anthropic among such clients and suggesting Meta may follow.
What to Watch
- Progress on the published roadmap: dynamic shapes, compilation performance, operator coverage, reinforcement learning and quantization.
- Whether Meta follows Anthropic in deploying Google TPUs on premises, which would give the partnership a commercial dimension beyond shared code.
- Adoption signals from developers currently running PyTorch on GPUs, since the value of TorchTPU depends on how many workloads migrate with only the device and backend names changed.
Checked
12 claims in this story checked against 1 source: 12 supported.
Show each claim
- SupportedGoogle and Meta open-sourced TorchTPU as a native PyTorch backend for Google TPUs.InfoQ: “TorchTPU is an open-source project developed by Google in collaboration with Meta that provides a native PyTorch backend for Google TPUs”
- SupportedTorchTPU takes a different approach from PyTorch/XLA and prioritises eager execution.InfoQ: “TorchTPU takes a fundamentally different approach, making eager execution a top priority”
- SupportedGoogle evaluated roughly 5,000 Hugging Face models with TorchTPU.InfoQ: “Google built an integration pipeline to evaluate about 5,000 Hugging Face models”
- SupportedTorchTPU lets existing PyTorch workloads run on TPUs with minor code modifications.InfoQ: “It enables PyTorch workloads to execute on TPU hardware with minimal code modifications”
- SupportedGoogle began enabling PyTorch workloads on TPUs in 2020 through PyTorch/XLA, which used JAX, XLA and lazy tensors.InfoQ: “Google began enabling PyTorch workloads to run on TPUs in 2020 through PyTorch/XLA, which leveraged JAX, XLA, and lazy tensors.”
- SupportedGoogle said PyTorch/XLA required intrusive source changes, had severe compile-time latency, lacked native eager-mode support and lagged in support for emerging distributed-computing primitives.InfoQ: “According to Google, this approach had significant shortcomings: Crucially, PyTorch/XLA required intrusive source-code modifications to explicitly define compilable regions, suffered from severe compile-time latency, lacked support for PyTorch's native imperative (“eager”) execution mode, and lagged in supporting emerging distributed computing primitives within the PyTorch ecosystem.”
- SupportedGoogle claims DeferAndFuse can be at least 50% faster than Strict Eager while reducing peak HBM memory.InfoQ: “Google claims "can be 50% or more faster than Strict Eager while reducing peak HBM memory"”
- SupportedGoogle tested about 5,000 models across more than 500 architectures in eager and compiled modes on TPU generations starting with v5e.InfoQ: “evaluate about 5,000 Hugging Face models across 500+ architectures in both eager and compiled modes on TPU hardware generations starting from v5e.”
- SupportedFor Muse Glimmer inference on one TPUv7 chip, throughput was 25.7k tokens per second at sequence length 5,120 and 22.4k at 10,240.InfoQ: “On a single TPUv7 chip, inference throughput reached 25.7k token/s at 5120 sequence length and 22.4k token/s at 10,240 sequence length.”
- SupportedFor Muse Glimmer training on four TPUv7 chips, throughput was 5.6k tokens per second at sequence length 5,120 and 4.6k at 10,240.InfoQ: “For training, using four TPUv7 chips, throughput reached 5.6k token/s and 4.6k token/s at 5120 and 10,240 sequence length, respectively.”
- SupportedGoogle used Qwen3 MoE 235B for training experiments scaling from 1,024 to 8,192 TPUv7 chips.InfoQ: “Google also conducted experiments using Qwen3 MoE 235B for training workloads, scaling from 1,024 to 8,192 TPUv7 chips.”
- SupportedHacker News user tingkuanpei wrote, “Very interesting work. Google should have done this a long time ago. PyTorch's ecosystem is just so rich.”InfoQ: “Hacker News user tingkuanpei observed: Very interesting work. Google should have done this a long time ago. PyTorch's ecosystem is just so rich.”
