Mark Heaps, Groq | theCUBE + NYSE Wired Media Week - Cyber & AI Innovators Summit
summary
Update on Groq’s inference platform: the company claims jump from 60 to 650,000 developers on its cloud API, pushes ultra‑low latency inference with the language processing unit LPU optimized for tokens per second and first‑token time, ships full racks for large deployments including racks shipped to Saudi Arabia, and recommends cloud API first then scale to on‑prem for tens of millions of users. Demonstrated real‑time apps—transcription, voice conversational agents, recommendation/dating prototype—and emphasizes cost and efficiency at scale.
AI-written summary · about
