Google TPU vs Nvidia GPU: What Meta's TorchTPU Changes - Memeburn

Google open-sourced TorchTPU at the PyTorch Conference on October 20, 2026, a backend developed with Meta's PyTorch team that enables PyTorch to treat Google's Tensor Processing Units (TPUs) as a native device alongside Nvidia GPUs and standard processors. Meta has confirmed that a serving stack built on TorchTPU is already running in production environments.

Google's pitch simplifies the migration process, allowing teams to move models by changing .to(cuda) to .to(tpu) in code. However, the PyTorch Foundation's post acknowledges that one cannot achieve peak TPU performance with that simple change. The real shift is in the cost of experimenting with TPUs rather than determining which chip is inherently faster.

Compatibility and Performance

TorchTPU supports approximately 450 core PyTorch operations and works in both step-by-step execution mode and with torch.compile. It has been validated with vLLM and SGLang serving engines and the TorchTitan training library. The serving component, SGLang-TorchTPU, was built by RadixArk's SGLang team with Google and Meta.

Compatibility testing shows TorchTPU ran 4,821 of 4,917 popular Hugging Face models without crashing (98% pass rate) and matched GPU numerical results in 638 of 657 architecture tests (97%). For production workloads seeking peak performance, custom kernels written in Pallas (Google's TPU kernel language) are still recommended.

Cost vs Latency Trade-offs

Independent benchmarks reveal Google's Ironwood chip serving tokens more cheaply than Nvidia's Blackwell GPUs at moderate speeds, while taking more than twice as long as Nvidia's B300 to return the first token under heavy load. This makes the workload, not the brand, the deciding factor.

SemiAnalysis testing shows Ironwood wins on cost and edges ahead on throughput by about 5%, but at 256 concurrent users it takes 1.4 times as long as the B200 and 2.25 times as long as the B300 to return the first token—the wait a chatbot user notices. Software improvements are rapidly changing the economics, as vLLM-TorchTPU's lowest cost for Qwen 3.5 fell 44% in about six weeks without new hardware.

Meta's Multi-Chip Strategy

Meta's TPU spending exists alongside substantial commitments to Nvidia and AMD. In February 2026, Meta signed a multi-year, multibillion-dollar deal to rent Google TPUs and was in talks to purchase TPUs for its own data centers. Simultaneously, it agreed to buy current and future Nvidia chips, while AMD announced it would sell Meta up to $60 billion in AI chips.

TorchTPU enables Meta to maintain software that allows the same PyTorch model to run across multiple suppliers. Google's TPU supply chain also involves Broadcom, which signed a long-term agreement covering future TPU generations. This positions Meta to leverage chips from multiple vendors while maintaining a unified software approach.

Where Nvidia Still Leads

Nvidia's CUDA software has matured over more than a decade, giving it an advantage in adoption. TorchTPU reveals limitations: a TPU can be accessed by only one process at a time, so reinforcement learning setups sharing a chip between training and serving aren't supported out of the box. Latency-sensitive serving, FP4 inference, and workloads built around hand-tuned CUDA kernels still favor Nvidia.

For infrastructure teams, the recommendation is to use TPUs for batch inference, overnight evaluation, and background agent runs where nobody waits on the first token, while keeping interactive chat on Blackwell GPUs. This approach leverages TPUs' cost advantages for certain workloads while maintaining Nvidia's strengths in low-latency applications. The landscape could shift with TPU v8i, expected to add native FP4 and potentially close the time-to-first-token gap.