NVIDIA Unveils Rubin CPX: A Purpose-Built GPU for Million-Token Context AI Inference

NVIDIA has announced the Rubin CPX — a dedicated GPU class built from the ground up for long-context AI inference at 1 million tokens and beyond. This is a meaningful architectural departure: every GPU NVIDIA has shipped at scale to date has been optimized primarily for training throughput. Rubin CPX flips that priority.

What the chip does differently

The Rubin CPX delivers up to 30 petaflops of compute at NVFP4 precision, paired with 128 GB of cost-efficient GDDR7 memory per GPU. The critical distinction is what sits alongside the compute: dedicated video encode/decode engines and a long-context inference pipeline that processes multi-million-token sequences without the KV-cache memory pressure that currently limits inference efficiency on training-oriented chips like the H100 and the standard Blackwell B200.

At the platform level, the Vera Rubin NVL144 CPX rack packs 8 exaflops of AI performance and 100 TB of fast memory. NVIDIA projects $5 billion in token revenue potential for every $100 million of infrastructure deployed — a framing aimed squarely at hyperscalers and AI cloud providers building inference capacity, not research labs buying training clusters.

Availability is planned for the end of 2026.

Why this matters for the AI stack

The context-length bottleneck is currently the most pressing constraint in production AI deployment. Models like GPT-5, Gemini Ultra, and Claude Opus can natively process contexts in the hundreds of thousands of tokens. The infrastructure to do so efficiently at scale doesn’t exist yet — most production inference still runs on chips and serving frameworks optimized for short-context batch workloads.

Rubin CPX directly targets three workload categories where long context is non-negotiable: long-format video generation (where coherent scene continuity requires holding thousands of frames in attention), large-scale code reasoning (where an entire repository may need to be in context simultaneously), and agentic AI workflows (where the model accumulates context across hundreds of tool calls and observations).

The GDDR7 choice — rather than the HBM memory used in training-focused chips — is a deliberate cost tradeoff. HBM delivers higher bandwidth but at significantly higher cost per GB. For inference workloads where you need enormous capacity but can tolerate lower peak bandwidth, GDDR7 reduces the $/token for operators dramatically.

Competitive position

Google’s TPU v5e and Amazon’s Trainium 2 both target inference efficiency, but neither has announced a chip architected specifically around context length as the primary design axis. AMD’s MI350 series focuses on training/inference balance. If Rubin CPX delivers on its specifications, NVIDIA has an 18-month window before a competitive response can reach production scale.

For teams building long-context inference infrastructure today: the constraints are real but temporary. The H100 and Blackwell B200 with careful KV-cache offloading strategies can handle 1M-token workloads at reduced throughput. Rubin CPX is the first chip where context length is a first-class design constraint rather than an afterthought.