From Paths to Dimensions: How We...
summary
Booking.com migrates ~20 million Graphite time series from a heterogeneous stack to Grafana Mimir using double writing and a runtime routing layer; it deploys 15k OpenTelemetry collectors (one per DB host), scrapes a verbose MySQL exporter (~900 metrics/host) and hits cardinality limits that require tenant tuning, ingestion-rate increases and custom drop filters. Dashboards shrink from 181 to 32 to migrate, and AI reduced manual work by 85% but manual verification remains.
AI-written summary · about
chapters (17)
- 0:00Introduction and agenda
- 1:04MySQL infrastructure at scale
- 2:28Why migrate from Graphite
- 4:52Migration challenges overview
- 5:54Five-phase migration plan
- 7:18Old stack architecture
- 9:05New OpenTelemetry stack
- 10:29Onboarding Badman to OpenTelemetry
- 11:30Migrating MySQL exporter
- 12:33Cardinality issues and fixes
- 14:41Migrating consumers introduction
- 16:51Metrics query router for Badman
- 17:55Grafana dashboards inventory
- 20:04Dashboards migration process
- 21:51AI tools for dashboard conversion
- 23:59Lessons learned summary
- 25:50Q&A session
