Meta deploys Google TPUs for AI inference, deepening Nvidia rivalry

PyTorch-native engines bring Google's custom chips into Meta's production workloads

By LineZotpaper

Published22 October 2026

Read Time3 min

Meta has confirmed it is running Google's tensor processing units (TPUs) in production for AI inference, using new PyTorch-native versions of the SGLang and vLLM inference engines. The move, announced on Google's developer blog, follows a three-way collaboration between Meta, Google and startup RadixArk, and comes a day after Amazon open-sourced its own PyTorch backend for its Trainium chips. Both developments aim to loosen Nvidia's software lock-in on AI hardware.

Google's custom AI chips, the tensor processing units (TPUs), are now serving AI workloads in production at Meta through a new PyTorch-native version of the SGLang inference engine, according to an announcement on Google's developer blog. A matching version of the vLLM inference engine was also announced as open source. Previously, both engines used Google's JAX AI framework to run on TPUs.

Meta's PyTorch-TPU lead Zongwei Zhou said SGLang-TorchTPU is "running in production at Meta today" after "a close three-way collaboration between Meta, Google, and the SGLang team." The team is based at RadixArk, a startup founded by SGLang's creators. Zhou described the engine as giving Meta "a remarkably mature, PyTorch-native serving stack."

Jana van Greunen, director of PyTorch engineering at Meta, said the company "cannot be constrained by a single hardware architecture" and that TorchTPU "enables us to deploy our PyTorch workloads to Google TPUs." Google said it co-developed TorchTPU with Meta's PyTorch team.

Google called PyTorch/XLA, the older bridge from PyTorch to TPUs, a "predecessor" to the new TorchTPU. SemiAnalysis reported last year that Meta research teams used TPUs through that bridge from 2020 to 2023 before canceling those contracts. While it is unclear where Meta's production TPUs are located, reports last year indicated talks about Google Cloud TPU rentals in 2026 with purchases in 2027, and Meta later reportedly signed a multi-year rental agreement.

The announcement came one day after Amazon said it would open-source TorchNeuron, its own native PyTorch backend for its Trainium2 and Trainium3 chips, at the PyTorch Conference.

On performance, a tuned vLLM setup serving Moonshot AI's open-weight, 2.8-trillion-parameter Kimi K3 model reportedly tops 700 tokens per second for a single user on the seventh-generation Ironwood TPU, according to Inferact, the startup founded by vLLM's creators that helped Google build vLLM-TorchTPU. That test ran 16 Ironwood chips with speculative decoding, compared to vLLM's published setup on 16 Nvidia GB200s, which reached 452 tokens per second. Kernels for Kimi K3 and Qwen 3.8 27B deliver roughly 1.4 to 2 times the GB200 baseline's throughput at small batch sizes without speculative decoding.

SGLang-TorchTPU will keep adding models, starting with DeepSeek V4.1 Flash and Reflection Beam. RadixArk said it is working toward Day 0 TPU support for new open models by the first quarter of 2027. Google, Meta and RadixArk plan to share more at SGLang Summit 2026 in San Francisco on Nov. 12-13.

Analysis

Why This Matters

The deployment of Google TPUs at Meta signals a major shift in AI hardware supply chains, breaking Nvidia's near-monopoly on high-performance AI inference.
PyTorch-native TPU support reduces barriers for other companies to adopt Google's custom chips, potentially accelerating competition in the data center AI market.
Interoperability between large AI players could lower costs and increase resilience for AI workloads, affecting businesses building on these platforms.

Background

The AI chip market has been dominated by Nvidia, whose CUDA software platform has locked in developers. Google's TPUs, now in their seventh generation (Ironwood), are custom ASICs designed for machine learning. Meta, which uses the PyTorch framework widely, has explored alternatives to Nvidia hardware for years. The new TorchTPU bridge allows PyTorch workloads to run on TPUs with minimal code changes, building on the older PyTorch/XLA interface that Meta research teams used earlier this decade. Amazon's similar move with TorchNeuron for its Trainium chips underscores the broader industry push for hardware diversity.

Key Perspectives

Meta: Seeks to avoid vendor lock-in and gain flexibility in hardware selection for its large-scale AI inference. By using TPUs, it can distribute workloads across architectures without retooling its PyTorch stack. Google: Expands the market for its TPU hardware beyond its own cloud services, positioning Ironwood as a viable alternative to Nvidia. The collaboration gives Google a marquee customer in Meta. Critics/Skeptics: The performance gains over Nvidia's GB200 (1.4-2x in certain configurations) are based on benchmarks from affiliated startups and may not reflect real-world production conditions. Nvidia's scale and continuous software improvements remain competitive advantages.

What to Watch

Whether other major AI companies (e.g., OpenAI, Anthropic) adopt similar PyTorch-TPU integrations, signaling broader market acceptance.
The outcome of the SGLang Summit 2026, where further technical details and roadmap for Day 0 TPU support will be shared.
Nvidia's response: potential pricing adjustments, new software features, or open-source moves to counter the erosion of its CUDA moat.

Sources

Google’s TPUs for AI workloads now run in Meta's data centers · Latest from Tom's Hardware

Checked

12 claims in this story checked against 1 source: 8 supported.

Show each claim

Not in the sourcesMeta's TPU deployment is deepening its rivalry with Nvidia.Latest from Tom's Hardware: “Both moves appear to be aimed at loosening Nvidia’s software lock-in .”
Partly supportedPyTorch-native versions of both SGLang and vLLM are bringing Google's TPUs into Meta's production workloads.Latest from Tom's Hardware: ““SGLang-TorchTPU is running in production at Meta today””
SupportedZongwei Zhou said SGLang-TorchTPU is “running in production at Meta today.”Latest from Tom's Hardware: ““SGLang-TorchTPU is running in production at Meta today””
SupportedZongwei Zhou described the work as “a close three-way collaboration between Meta, Google, and the SGLang team.”Latest from Tom's Hardware: “after “a close three-way collaboration between Meta, Google, and the SGLang team,” said Meta’s PyTorch-TPU lead Zongwei Zhou.”
SupportedZhou said the engine gives Meta “a remarkably mature, PyTorch-native serving stack.”Latest from Tom's Hardware: “The engine, he said, gives Meta “a remarkably mature, PyTorch-native serving stack.””
SupportedJana van Greunen said Meta “cannot be constrained by a single hardware architecture.”Latest from Tom's Hardware: “Jana van Greunen, director of PyTorch engineering at Meta, added in Google’s TorchTPU post that Meta “cannot be constrained by a single hardware architecture””
SupportedVan Greunen said TorchTPU “enables us to deploy our PyTorch workloads to Google TPUs.”Latest from Tom's Hardware: “and that TorchTPU “enables us to deploy our PyTorch workloads to Google TPUs.””
SupportedGoogle called PyTorch/XLA a “predecessor” to TorchTPU.Latest from Tom's Hardware: “Google calls PyTorch/XLA, the older bridge from PyTorch to TPUs, a "predecessor" to the new TorchTPU”
Partly supportedAmazon open-sourced TorchNeuron a day before the Meta announcement.Latest from Tom's Hardware: “The announcement came one day after Amazon said it was open-sourcing TorchNeuron, its own native PyTorch backend for its Trainium2 and Trainium3 chips, at the same PyTorch Conference.”
Partly supportedBoth developments aim to loosen Nvidia's software lock-in.Latest from Tom's Hardware: “Both moves appear to be aimed at loosening Nvidia’s software lock-in .”
SupportedA tuned vLLM setup on 16 Ironwood TPUs reportedly tops 700 tokens per second for one user, compared with 452 tokens per second on 16 Nvidia GB200s.Latest from Tom's Hardware: “Its test ran 16 Ironwood chips with speculative decoding, a method of speeding up large language model inference with a small draft model, against vLLM’s published setup on 16 Nvidia GB200s, which reached 452 tokens per second.”
SupportedKernels for Kimi K3 and Qwen 3.8 27B deliver roughly 1.4 to 2 times the GB200 baseline's throughput at small batch sizes without speculative decoding.Latest from Tom's Hardware: “Its kernels for Kimi K3 and Qwen 3.8 27B deliver roughly 1.4 to 2 times the GB200 baseline’s throughput at small batch sizes without speculative decoding.”