How We Built DBRX - An Open High-quality LLM
summary
Databricks builds DBRX as an open foundation LLM trained on 12 trillion curated tokens using Spark, MLflow, LLM Foundry and Megablocks, using a sparse Mixture-of-Experts design with 132B total params, 36B active and support for 32k token context. Training uses 3,000+ H100s over ~3 months. DBRX serves at ~150 tokens/sec, outperforms many open models on MML and HumanEval, beats GPT‑3.5 on several tasks, and is competitive with Gemini 1.0 Pro, with special strength on code and long‑context RAG.
AI-written summary · about
chapters (12)
- 0:00DBRX introduction
- 1:03What is DBRX
- 4:54Why we built DBRX
- 8:45DBRX model architecture
- 13:05Research and optimizations
- 17:42Data processing and training
- 22:16DBRX performance and use cases
- 26:15Serving DBRX efficiently
- 29:06How to try DBRX
- 31:55Live demo on laptop
- 35:03Summary and future
- 37:10Q&A
related talks
41mMeet DBRX, the New Standard for High-quality LLMs, and How...
16mKeynote: DBRX: Training Modern LLMs on Databricks
40mTechniques to avoid common LLM pitfalls
18mElevating enterprise data through open source LLMs
05mBuild your own LLM on Cloud
29mLeveraging open-source LLMs for production
28mDatabricks' vLLM Optimization for Cost-Effective LLM Inference
22m360Brew: LLM-based Personalized Ranking and Recommendation