Live Q&A
summary
Meta-style infra practices get applied to ML: resilience, incident management, clear component contracts and efficiency; new needs include batch GPU scheduling, tenant management, numerical model debugging and orchestrated maintenance and handling new hardware-stack bugs. Product-side specifics: one-stop model marketplaces that surface top models and distillation paths (Llama 3.1 cited) to move from large to cheaper small models; internal tools include Code Llama completions and Metam Mate; testing favors dogfooding, manual end-to-end tests, premortems and smoke tests; Roblox uses ~15 million experiences/year for training data.
AI-written summary · about
chapters (12)
- 0:00Introduction
- 1:07Infrastructure lessons for AI
- 2:31Model fatigue in AI
- 4:17Med AI tools and experts
- 5:21AI in Med AI code
- 6:27When to ship AI features
- 8:13Balancing AI paradigms
- 9:57Small vs large AI models
- 15:19Risks and deadlines in AI
- 18:30Testing AI features
- 20:35Scaling AI data at Roblox
- 23:03Closing
related talks
26mML Product Experiments at Scale
42mImproving Machine Learning Development Reliability
20mScaling Meta’s Infra with GenAI: Journey to faster and smarter Incident Response
20mLive Q&A 3 with Speakers
32mTeam Teardown: Organizing and Supporting ML
33mLive Q&A Session 2
27mLive Q&A Session 1
24mEngaging new and diverse audiences with ML