Amazon releases TorchNeuron to pull AI workloads off Nvidia chips - Startup Fortune

Amazon announced TorchNeuron at AWS re:Invent in December 2025, shipping an initial private beta as part of the Neuron 2.27 SDK release under the Apache 2.0 license on GitHub. The tool allows developers to run the same PyTorch code they would use on Nvidia GPUs on Trainium2 or Trainium3 chips by changing just a configuration line rather than rewriting their entire training stack.

Amazon has spent years trying to convince developers that its custom silicon is a credible alternative to Nvidia. TorchNeuron represents the clearest attempt yet to remove the excuse that switching between platforms requires too much engineering work.

The Nvidia moat

Nvidia's dominance in AI computing isn't just about its chips—it's the CUDA software layer and the ecosystem built around it over nearly two decades, including cuDNN for neural networks and NCCL for multi-GPU communication. This has created significant switching costs measured in engineering weeks rather than just dollars.

AWS Neuron, the software layer behind Trainium, has historically lagged behind this ecosystem. Its slow support for fully sharded data parallel (FSDP), a standard technique for splitting large models across GPUs, has been widely cited as a reason major AI labs were hesitant to adopt Trainium for training.

Technical breakthrough

TorchNeuron represents a fundamental rewrite of how Trainium integrates with PyTorch. The previous integration, torch-neuronx, ran through XLA, a compilation layer that forced code into a graph before execution, making debugging difficult and breaking compatibility with parts of the PyTorch ecosystem.

The new tool drops XLA entirely and integrates through PyTorch's native PrivateUse1 device backend, enabling eager-mode execution for easier debugging and standard distributed APIs like FSDP and DTensor without extra plumbing. AWS claims models built on HuggingFace Transformers or TorchTitan need minimal code changes to run on it.

Major commitments

Amazon already has significant customers committed to using Trainium. Anthropic has pledged to spend more than $100 billion over ten years on AWS capacity, built around Project Rainier—a cluster that went live in late 2025 with 500,000 Trainium2 chips dedicated to training Claude, scaling toward roughly 1 million chips.

OpenAI's deal with Amazon, initially valued at $38 billion but potentially growing toward $100 billion, includes a commitment to consume roughly 2 gigawatts of Trainium capacity. These represent the two most compute-hungry AI labs in the world agreeing to run serious workloads on Trainium chips.

The economic argument for Trainium has been compelling for some time, with AWS pricing Trainium2 instances at roughly half the cost of comparable Nvidia H100 capacity for similar performance on many workloads—a significant difference at the scale Anthropic and OpenAI operate.

The real test

While TorchNeuron removes a major barrier to adoption, its success will ultimately be determined by whether mid-size AI companies—not just Amazon's anchor tenants like Anthropic and OpenAI—decide the switching cost is now low enough to actually try it on real training runs.

AWS has made TorchNeuron free and open source, lowering the barrier as much as possible without paying companies to switch. Whether this is enough to overcome the software ecosystem Nvidia has spent twenty years building will be the key question to watch over the next few quarters.