9 Lessons Learned from Deploying GenAI at Scale
summary
Liberty Mutual scales GenAI across thousands of engineers by building a centralized platform, dashboards and cost controls while exposing LibertyGPT and GitHub Copilot to teams; adoption hits about 70% and 60→90% respectively inside Liberty IT. Operations support a plurality—around 28 models—and a bad model choice once threatens $40k before switching saved the program. RAG proves brittle (off‑the‑shelf stacks ~65% effective), latency drops from 15s to 3s after GPU and UX fixes, and ongoing iteration replaces many short‑lived homegrown services.
AI-written summary · about
chapters (14)
- 0:00Intro
- 2:16GenAl adoption was a roller coaster
- 4:221. Learning never stops
- 7:432. Shipping isn't adoption
- 16:233. Models aren't equal
- 19:104. Scaling is a beast
- 24:395. Today's fix is tomorrow's pain
- 27:486. Stay out of the weeds
- 30:387. Costs can spiral
- 35:158. RAG is really hard
- 40:59Latency is real
- 42:52Summary
- 43:54Call to action
- 44:56Outro
related talks
34mOrganized chaos adopting GenAI at Enterprise
28mUnlocking the power of GenAI
36mGenAI governance in practice
29mBeyond access: Building the GenAI platform behind 3000+ developers
24mScalable GenAI in Production
23mScaling GenAI Applications and Agentic AI for Business Competitive Differentiation- 29mBeyond access: Building the GenAI platform behind 3000+ developers
36mUnleashing Generative AI: Improving Developer Productivity in SRE