Platform Agnositc Observaility System for AI Accelerators
summary
Facebook reports running about 150 trillion AI predictions per day and processing 250 million video uploads daily. It builds an observability stack: asicmon provides a uniform, lightweight accelerator abstraction across PCIe/M.2 devices; Asimov compiles spec files into C++ modules to cut onboarding from months to weeks; and atrace collects correlated CPU/accelerator traces linked into Glow and PyTorch. Concrete wins include automatic scaling from device utilization and closing up to 10% of a PyTorch vs Caffe2 performance gap.
AI-written summary · about
chapters (12)
- 0:00AI and video accelerator intro
- 1:10AI accelerators history
- 3:21Facebook's AI hardware
- 5:53AI accelerator server design
- 6:36Why observability matters
- 7:39ASICMon observability framework
- 9:44ASIMO spec language
- 11:27ASICMon and ASIMO use cases
- 12:50Debugging performance issues
- 14:34Tracing for AI model optimization
- 15:57CPU and accelerator correlation
- 17:01Future tracing and observability
related talks
22mAI Observability At Meta Scale
40mOptimizing AI Applications for Production with Observability
44mAgentic Observability — The Path to AI Co-SRE
38mAI Monitoring Mastery: Optimize Performance, Ensure Compliance, and Cut Costs
47mValidation and Observability of AI Agents
29mNon-Invasive AI Agent Observability With OBI
28mDO WE STILL NEED TO "OBSERVE" THE FUTURE OF AI & OBSERVABILITY
30mBuilding Reliable AI Agents: An Open Source Approach To Evaluation and Observability