Google's TurboQuant Crashed the AI Chip Market

TL;DR

Google's TurboQuant algorithm cuts AI memory use 6x and speeds inference 8x with zero accuracy loss, tanking memory chip stocks.

Key Points

• 1.TurboQuant achieves a 6x reduction in KV cache memory and 8x speed increase with zero accuracy loss. Tested on real Nvidia H100 GPUs running open-source models including Gemma, Mistral, and Llama, with no model retraining or fine-tuning required.
• 2.The KV cache is the core mechanism TurboQuant optimizes. As LLMs read text, they store word relationships in key-value cache 'folders' — TurboQuant compresses this cache using polar coordinates instead of step-by-step Cartesian directions, pointing directly at meaning in vector space.
• 3.TurboQuant has two components: Polar Quant and the Quantized Johnson-Lindenstrauss algorithm. Polar Quant does the heavy lifting on compression and speed; QJL uses just one residual bit to eliminate any leftover errors, achieving the zero accuracy loss claim.
• 4.The practical cost impact for enterprise AI is approximately 50% cheaper inference. Every prompt, API call, chatbot response, and agentic workflow benefits immediately, with no changes to the underlying model — it also expands effective context windows on existing hardware.
• 5.Memory chip stocks crashed on the news, with SK Hynix down 6%, Samsung down 5%, Micron down 3%, and SanDisk and Western Digital each down roughly 5%. Market logic: if AI needs 6x less memory, demand for DRAM and storage chips falls.
• 6.The Jevons Paradox argues the stock selloff is likely overblown. When compute efficiency rises and costs drop, use cases multiply rather than contract — cheaper inference typically drives more AI adoption, not less GPU and memory demand.
• 7.Google published TurboQuant openly, continuing a pattern of releasing foundational AI research despite the competitive advantage of keeping it secret. Previous examples include the 'Attention Is All You Need' transformer paper, which spawned most current AI competitors — a move the video argues deserves recognition.