Monitoring Cloudflare's Planet-Scale Edge Network
summary
Cloudflare runs Prometheus on each POP, federates selected metrics to four HA top-level servers, and reports concrete scale: 116 locations, 188 Prometheus instances, 4.8M time series with a single server ingesting 72,000 samples/sec. POP servers scrape every 60s and federated tiers every 30s, retain 15 days, and use node_exporter, blackbox, mtail and cadvisor. Alerts flow to Alertmanager, ChatOps and Jira; custom tools include a pager drill and the unsee open source alert dashboard.
AI-written summary · about
chapters (14)
- 0:00Start
- 0:57Key Takeaways
- 1:53What Is Prometheus
- 2:29Why I Might Use Prometheus
- 5:05Prometheus Deployment
- 8:11Prometheus Queries in the Data Model
- 9:24Metrics for Alerting
- 10:38Prometheus Architecture
- 13:41How Do You Deploy Prometheus
- 15:29Prometheus Node Exporter
- 16:31Federation
- 19:34Namespace Metrics
- 20:26Alerting
- 28:22What Are the Lessons Learned
related talks
17mMonitoring DNS with Open-Source Solutions
32mCDN Scaling Challenges
46mLessons learned from monitoring a large production network
14mSkyline: A Cloud Centric Internet Monitoring Engine
23mGlobal Scale - Geo-Distributed Ingress
24mDeep Network Traffic Observability with Pktvisor and Prometheus
31mCloud Native Network Observability: Treating Routers like Kubernetes Resources
16mNetwork Flow Data in the Cloud