Meta Deploys Google TPUs: A New Native Path for AI Inference

Google has officially enabled its custom tensor processing units to run natively in Meta's data centers. The tech giant announced a new PyTorch-native version of the SGLang inference engine, marking a significant shift in how Meta handles its artificial intelligence workloads. This move allows Google's hardware to function as a native option for PyTorch developers without the need for extensive code rewriting.

The announcement highlights a major collaboration between the two industry leaders. Zongwei Zhou, Meta's PyTorch-TPU lead, confirmed that the SGLang-TorchTPU system is already running in production at Meta. He described the achievement as the result of a close three-way partnership involving Meta, Google, and the SGLang team. Zhou noted that this new engine provides Meta with a remarkably mature and PyTorch-native serving stack.

Previously, developers had to use Google's JAX framework to run workloads on TPUs. The new software versions change this dynamic by allowing the hardware to integrate directly with PyTorch. This removes a significant technical barrier for companies that have built their AI infrastructure around the PyTorch ecosystem. Meta's PyTorch engineering director, Jana van Greunen, emphasized that the company cannot be constrained by a single hardware architecture. She stated that the new TorchTPU backend enables Meta to deploy its PyTorch workloads directly to Google TPUs.

This is not the first time Meta has experimented with Google's silicon. Google identifies its older PyTorch/XLA bridge as a predecessor to the new TorchTPU system. Reports from SemiAnalysis indicated last year that Meta research teams used TPUs through this older bridge between 2020 and 2023. However, those initial contracts were eventually canceled. The current push appears to be a more robust solution aimed specifically at Meta, which had previously resisted migrating its core operations to Google's JAX framework.

The physical location of these production TPUs remains somewhat unclear. However, industry reports from last year suggested that negotiations included Google Cloud TPU rentals for 2026, with potential purchases scheduled for 2027. Meta is also reportedly signed to a multi-year rental agreement. This arrangement suggests a long-term commitment to integrating Google hardware into Meta's broader AI infrastructure.

The timing of this release is strategically significant. It came just one day after Amazon announced it would open-source TorchNeuron, its own native PyTorch backend for Trainium2 and Trainium3 chips. Both announcements were made at the same PyTorch Conference. Analysts view these parallel moves as a concerted effort to loosen Nvidia's software lock-in, which has historically dominated the AI chip market through its CUDA ecosystem.

Performance benchmarks suggest the new setup is highly competitive. A tuned vLLM configuration serving Moonshot AI's Kimi K3 model reportedly achieves over 700 tokens per second for a single user. This speed was recorded on the seventh-generation Ironwood TPU. The test involved 16 Ironwood chips using speculative decoding, a technique that uses a smaller draft model to speed up inference. This result outperformed a published setup using 16 Nvidia GB200 chips, which reached 452 tokens per second.

Further testing by Inferact, a startup founded by the creators of vLLM, showed that kernels for Kimi K3 and Qwen 3.8 27B deliver roughly 1.4 to 2 times the throughput of the GB200 baseline. These figures apply to small batch sizes without speculative decoding. The results indicate that Google's hardware can match or exceed Nvidia's performance in specific inference scenarios, provided the correct software optimization is applied.

The ecosystem is expected to expand rapidly in the coming months. SGLang-TorchTPU plans to add support for more models, starting with DeepSeek V4.1 Flash and Reflection Beam. RadixArk, the startup behind the engine, is also working toward Day 0 TPU support for new open models. This means the software will support new releases on their launch day, a goal targeted for the first quarter of 2027. Google, Meta, and RadixArk are scheduled to share more details at the SGLang Summit 2026 in San Francisco on November 12 and 13.