Meet DBRX, the New Standard for High-quality LLMs, and How...
summary
Databricks unveils DBRX as an open source mixture-of-experts LLM claiming 132B parameters with 36B active and up to 3x faster inference; it trains on 12 trillion tokens using curriculum learning, ran ~40 days on ~3,100 H100s, and uses a 16‑choose‑4 expert routing with a dropless implementation to avoid expert starvation. The stack stitches Mosaic/MosaicML, Composer, Megablocks, PyTorch FSDP, Lilac for data exploration, MLflow for tracing and LM‑as‑judge evaluation, and a Vector Search service that demoed end‑to‑end deployment, logging, and user feedback.
AI-written summary · about
chapters (14)
- 0:00DBRX introduction
- 1:02Why build custom LLM
- 3:05DBRX architecture details
- 8:48Training on Databricks
- 13:00Mixture of Experts design
- 16:56Open-source tools
- 21:33When to train custom LLM
- 24:46Demo and platform
- 25:28Building RAG pipelines
- 31:26Tracking LLM chains
- 32:47User feedback evaluation
- 36:01UI with Databricks Agents
- 37:48Governance and vectors
- 40:39Q&A
related talks
43mHow We Built DBRX - An Open High-quality LLM
28mDatabricks' vLLM Optimization for Cost-Effective LLM Inference
42mDataEngBytes 2023 - SYD-T1-02 - Mikiko Bazeley
49mGenAI for Business on the Modern Data Stack
47mGive Your LLMs a Left Brain
33mDeep research at scale
28mNavigating LLM deployment: Tips, tricks, and techniques
01h 49mAI engineering with open access LLMs that lie, curse, and steal