Scaling Machine Learning at Discord: Our Journey with Ray and Anyscale
summary
Discord moves from single-node XGBoost to a distributed multitask neural ranker on Ray and Anyscale, shipping a production model that trains on 2 billion raw events down to ~30 million records and completes training and scoring in about an hour. Implementation uses Ray Data/Training, PyTorch with jagged tensors and sharded embedding bags across eight H100s, plus a multi-gate mixture-of-experts; swapping to the Anyscale operator yields 40% faster runs, some teams report 90% compute reduction, and serving hits 18 ms median latency.
AI-written summary · about
chapters (10)
- 0:00Introduction and team overview
- 1:54Initial Ray platform architecture
- 5:49Quest ML team's scaling challenges
- 9:03Shared multitask neural ranker design
- 13:40Ray platform 2.0 with Anyscale
- 17:16Migration experience and benefits
- 20:50Online model serving strategy
- 23:19Future ML innovations at Discord
- 25:10Key takeaways and team recognition
- 26:24Q&A session
related talks
21mHow Ray and Anyscale Make it Easy to do Massive-scale ML on Aerial Imagery
20mThe Quick Journey to Using Ray: How We Implement Ray and Anyscale to Speed up our ML Processes
27m[Opening Keynote] Anyscale Demo: Machine Learning Application from Dev to Prod
32mAnyscale's Ray Data: Revolutionizing Batch Inference
13mRay and Anyscale: An Optimization Journey
28mScaling Multimodal Data and Reinforcement Learning with Ray
21mStreamlining AI Workflows with Apache Airflow and Ray
44mPanel: ML Platform on Ray