Nvidia's Groq 3 LPX Hits Full Production: 3,400 Tokens/Second and a New Chip Class for AI Agents

Nvidia announced Monday that Groq 3 LPX, its dedicated low-latency inference accelerator, has entered full production. Built on the LPU (Language Processing Unit) architecture Nvidia absorbed with Groq, the chip is not a GPU — it is a purpose-built token-generation engine designed to bolt onto the Vera Rubin platform and make AI agents fast.

The headline benchmark: running Gemma 4 31B, Google’s open-source agentic model, a Groq 3 LPX system delivered 3,400 output tokens per second on 100,000-token long-context workloads — 4x faster than the nearest alternative platform, per Artificial Analysis. That long-context qualifier matters. Generating fast on a short prompt is easy; sustaining throughput when the model is dragging 100K tokens of agent history behind every decode step is the hard problem, and it is exactly the shape of real agentic workloads.

Division of labor, at rack scale

The architecture is explicit about specialization. Vera Rubin GPUs handle prefill — the parallel, compute-bound work of ingesting context. Groq 3 LPX handles decode — the serial, latency-bound work of generating tokens one at a time. A full system packs 32 liquid-cooled 1U trays, each with eight third-generation LPUs, for 256 accelerators per rack connected by direct chip-to-chip links.

Nvidia calls this “extreme codesign,” and it dissolves the classic speed-versus-throughput tradeoff: you no longer choose between a system tuned for batch efficiency and one tuned for interactive latency. For agents — which loop through generate, call tool, read result, generate again, dozens of times per task — decode latency compounds on every iteration. Cut it 4x and an agent that took two minutes finishes in thirty seconds.

Nebius is the first AI cloud to adopt the chip; CoreWeave is deploying the companion Spectrum-X networking. No pricing was disclosed.

SpaceX puts Vera CPUs in orbit

The companion announcement is stranger and arguably bigger: SpaceX’s AI arm will deploy Nvidia’s Vera CPUs to power the non-GPU half of agentic AI — orchestration, tool use, code execution, data processing, simulation — scaling “from Earth-based data centers to orbital satellites.” Compute in orbit has obvious appeal for a company already operating thousands of satellites with solar power and free cooling, but running agent orchestration off-planet is a genuinely new deployment target.

The strategic read: Nvidia spent two years watching inference-specialist startups — Groq before the acquisition, Cerebras, Etched — claim order-of-magnitude speed advantages on token generation. Cerebras’ CS-4 launched last week claiming 30x over GPUs. Groq 3 LPX is Nvidia’s answer, and it is characteristically thorough: rather than defending the GPU’s honor, Nvidia simply added the rival architecture to its own platform. The moat was never CUDA alone. It is that every credible answer to Nvidia now ships inside an Nvidia rack.

Sources

• NVIDIA Blog
• SiliconANGLE