Inside Amazon's Trainium Lab: The Chip Powering Anthropic, OpenAI, and Apple

Shortly after Andy Jassy announced AWS’s $50 billion investment deal with OpenAI, Amazon gave TechCrunch a private tour of the chip development lab behind that commitment. The facility sits on a high floor of a glass-and-chrome building in Austin’s Domain district — part corporate campus, part functional hardware lab, about the size of two large conference rooms and loud with cooling fans.

The numbers are striking. AWS has 1.4 million Trainium chips deployed across three generations. Anthropic’s Claude runs on more than 1 million of the Trainium2 chips alone. The new OpenAI deal puts AWS on the hook to supply 2 gigawatts of Trainium compute capacity — a commitment that comes while Anthropic and Amazon’s own Bedrock service are already consuming chips faster than they can be produced.

Trainium3, released in December, is a 3-nanometer chip manufactured by TSMC. AWS claims it costs up to 50% less to run than comparable conventional cloud servers for the same AI workload. For workloads generating trillions of tokens a day, that cost reduction is not trivial. AWS has also announced an integration with Cerebras Systems, pairing Cerebras inference chips with Trainium-based servers for low-latency AI.

The team’s director Kristopher King pointed out that Trainium was originally designed for faster model training — a major priority a few years ago — but is now tuned and used heavily for inference. Inference is the current industry bottleneck. Trainium2 already handles the majority of inference traffic on Amazon’s Bedrock, which serves enterprise customers building AI applications.

What makes Trainium a credible alternative to Nvidia’s H100 and B200 GPUs, beyond price, is software compatibility. Director of engineering Mark Carroll described the migration path: “basically a one-line change, and then recompile, and then run on Trainium.” The addition of PyTorch support — including models hosted on Hugging Face — removes what has historically been the biggest switching cost from Nvidia chips.

Amazon’s custom silicon team traces back to its 2015 acquisition of Israeli chip designer Annapurna Labs for roughly $350 million. Over a decade, that team produced Graviton (used and publicly praised by Apple), Inferentia (inference-optimized), and now three generations of Trainium. The approach is the classic Amazon playbook: observe what the market needs, then build a cheaper in-house alternative.

The OpenAI deal is the most significant public endorsement of that strategy. If AWS becomes the exclusive provider for OpenAI’s new Frontier agent-builder platform, Amazon needs to be able to deliver at a scale that even Nvidia has struggled to match quickly. The lab tour was clearly timed to signal that it can.