PyTorch Conference 2026: Google and Meta Open-Source TorchTPU for Native PyTorch on TPUs

TorchTPU replaces PyTorch/XLA, a lazy-tensor bridge from 2020 that, according to the announcement, "required intrusive source-code modifications." The new backend aims to make porting a model as simple as changing model.to("cuda") to model.to("tpu"). "With TorchTPU, TPUs join GPUs and CPUs as a native option for PyTorch," Google Cloud wrote in its announcement, co-authored by Bill Jia, VP of engineering for Core ML/AI.

Jia presented the keynote "Workload Fungibility in the Agentic Era: Bringing Native PyTorch to Cloud TPUs" with Zhou, after two talks that framed the demand. Mark Collier, executive director of the PyTorch Foundation, said inference has overtaken training as the largest data center workload, creating "an incredible hunger for tokens." Collier described the community's goal as open source software for models that "run on any chip in any cloud and connect to any agent." The Foundation's keynote slides showed how fast that market moves: DeepSeek's Flash model became 60% cheaper in 37 days, and Kimi K3 ran 68% faster on the same hardware in 33 days.

Mazin Gilbert, executive director of the Agentic AI Foundation, said agent token consumption passed human consumption on February 6, 2026. It has since grown 14-fold, against 2.8-fold for humans. "So this is basically a hockey stick that's exploding," said Gilbert. Practitioners should check the scope before reusing those numbers, because the public data comes from a single platform, OpenRouter.

Because TorchTPU plugs into PyTorch's low-level dispatcher, the speakers said, Hugging Face models worked with it out of the box. The vLLM and SGLang inference engines run natively, and TorchTitan supports dense, mixture-of-experts, and diffusion models for training at scale. Ray and llm-d scale TPU workloads across hosts on Kubernetes. Four repositories were open sourced: TorchTPU, vLLM-TorchTPU, SGLang-TorchTPU, and TorchTitan.

Jia showed near-linear scaling past 8,000 TPU chips, with per-chip throughput holding roughly constant as the cluster grew. Google's published results include a Qwen3 235B mixture-of-experts run on 8,192 TPU v7x chips at 28% model FLOPs utilization. The announcement is candid that the one-line change is only a starting point:

One cannot achieve peak TPU performance with that simple change, since a model's shapes, sharding and ops may be skewed toward a GPU architecture; however, the model should run out of the box.

Google's answer to that tuning gap is more agents. Jia closed by announcing the TPU agent suite, now in preview on GitHub, which plugs into coding harnesses to automate onboarding, diagnostics, and performance tuning.

Zhou said Meta's PyTorch team co-developed TorchTPU with Google "to make TPU a first-class native PyTorch target," citing production needs for performance, reliability, and maintainability. "All of this is open source and available today and it is just a beginning," Zhou said. Jana van Greunen, director of PyTorch engineering at Meta, gave the motivation in Google Cloud's announcement:

At Meta, advancing the frontier of AI requires immense compute scale and deep hardware diversity. We cannot be constrained by a single hardware architecture.

Google was not alone. AWS put TorchNeuron, a native PyTorch backend for its Trainium chips, into public beta on October 20, with.to('neuron') as the only code change. Both backends register through PrivateUse1, PyTorch's extension point for custom hardware. TorchNeuron's first release has gaps: The Next Web reported that it does not yet support dynamic input shapes and skips about 22,000 PyTorch tests. In an analysis for Forkast, Blair Hayes argued that the lock-in has moved rather than disappeared:

While the PyTorch extension point provides a common language for developers, the optimization and supply layers remain firmly under vendor control.

On Hacker News, where the TorchTPU launch thread reached 379 points, relief mixed with practical doubts. "I remember the days when you had to fight XLA to make PyTorch run on TPUs, it seems we finally got first-party support!" wrote one commenter. Others asked where they could buy a TPU at all. A commenter whose company had looked at Google's on-premises TPUs was cooler: "We've evaluated them but decided to stick with Nvidia, at least for one more generation."

Portability was also the pitch below the data center. In a session on deploying robot policies with ExecuTorch, Meta's Anthony Shoumikhin and Jacob Szwejbka argued that policies can stay in PyTorch from training to the device, with no conversion stack. The same quantization and operator graphs run on targets from TensorRT on NVIDIA Jetson to OpenVINO on Intel.

The training breakouts put numbers on the cost side. Presenting the Miles reinforcement learning framework, Yueming Yuan of RadixArk said fusing two log-probability computations into one kernel made that step more than 10 times faster. Delta updates cut the data moved per weight sync from 62 GB to under 1 GB in one large run.

Open tooling for training agents with RL ran through several sessions. Anyscale's SkyRL separates training, inference, and environments and orchestrates them with Ray. Hugging Face's OpenEnv, which joined the Agentic AI Foundation on October 22, gives every RL environment the same interface. Using TRL and OpenEnv, Hugging Face's Sergio Paniego Blanco trained a small Liquid AI model through four coding agents, raising its score on held-out tasks from 42% to 54%.

For teams weighing a move, the one-line device change gets a model running, not tuned, so budget for sharding and kernel work and confirm TPU capacity first. Developers interested in learning more can read the TorchTPU announcement, the TorchTPU repository, or InfoQ's coverage of PyTorch Conference 2024. Session recordings are expected on the PyTorch Foundation's YouTube channel within two weeks of the event.

Presented by: Martin Reynolds - Field CTO at Harness, and Barani Mohandoss - Director, Cloud Governance, FinOps & E-Tools at Trimble