"What Would I Do With 8 GPUs?" — 10 Years Later… Millions ft Charlie Boyle from Nvidia

TL;DR

Nvidia's Charlie Boyle reflects on 10 years of DGX evolution, from customers puzzling over 8 GPUs to million-GPU AI factories built around the new Vera Rubin platform.

Key Points

• 1.DGX1 launched 10 years ago to skeptical customers. The top question in year one was 'what am I going to do with 8 GPUs?' — the industry has since scaled to millions of deployed GPUs across global AI factories.
• 2.The AI factory concept evolved from the A100 Super Pod. Nvidia's DGX Super Pod — 32 A100s connected via Infiniband with partner storage and a performance guarantee — became the blueprint for today's full AI factory architecture.
• 3.Vera Rubin introduces a CPU-GPU recipe for agentic workloads. The new pod spec pairs one Vera CPU rack (256 Grace CPUs) with every eight Vera Rubin GPU racks, enabling the CPU-heavy unit testing and reasoning that agentic AI requires.
• 4.The Groq acquisition adds ultra-low-latency inference to the AI factory stack. The LPX Groq rack complements Vera Rubin for latency-sensitive, monetizable inference — serving fast responses to end users while Rubin handles large-scale compute.
• 5.STX is a new storage tier built for agentic data velocity. Agents can generate terabytes of temporary data per task at machine speed; STX provides a sandboxed, RDMA-accessible rack-level storage tier with partner-built software stacks and deny-all security policies via existing enterprise ACLs.
• 6.Grace Blackwell delivered 50–100x speedups over Hopper for real customer workloads. One customer's job that took 18–36 hours on a Hopper Super Pod runs in 20 minutes on Grace Blackwell, enabling proportionally more customers at lower cost.
• 7.Vera Rubin achieves EDPP1 flat power draw, unlocking near 100% data center utilization. New circuitry at the chip, rack, and PSU level allows software-settable power caps, raising usable provisioned power from ~60% toward 100%, and enabling utilities to remotely dial down non-critical workloads automatically — validated at Nvidia's planned gigawatt-scale DSX facility in Northern Virginia.