Jonathan Ross, Groq | SC23
summary
Groq describes an LPU built as a compiler‑first system rather than a single chip, running inference across a 576‑chip factory that pipelines work instead of relying on large batching. Claims include 10× faster, 10× cheaper, 10× lower power versus GPUs and often 20× faster software compile-to-run times; best performance at batch size 20. Target is inference (not training), deployed on 14 nm silicon assembled in the US, sold as tokens-as-a-service or hardware to finance, government, clouds and startups.
AI-written summary · about
