From Chaos To Clarity: Scaling Observability at Dropbox With Centralized Logging Solution
Talkat Observability Day 2025 · on Tuesday, 2025-04-01· at 14:05· Level 1 | Hall Entrance S10 | Room C·
summary
Dropbox moves from on-host SSH logs to a centralized Loki-based pipeline that ingests ~10 GB/s (150 TB/day baseline), stores ~10 PB for 30 days on an internal Magic Pocket object store, and exposes logs via Grafana. Goals: P99 ingest <30s, queries <10s, 99% durability and tenant RBAC with per-tenant rate limits and a “break glass” flow. Changes include disabling WAL for availability, switching to memberlist gossip for membership, and achieved lower cost, longer retention, and multi-tenant scale.
AI-written summary · about
chapters (10)
- 0:00Introduction
- 1:45Dropbox scale and logging
- 4:20Requirements for logging
- 8:58Logging solutions evaluation
- 11:07Why Dropbox chose Loki
- 13:37Loki indexing and storage
- 18:57Dropbox’s Loki architecture
- 20:48Scaling Loki at Dropbox
- 24:17Optimizing Loki for Dropbox
- 27:31Summary and results
related talks
28mFrom Fragmented Logs to Unified Insights: Scaling O11y
41mScaling Observability: From Chaos to Clarity with Open Source Tools
30mData @Scale Seattle- Sundaram Narayanan
40mLarge-Scale Logging Made Easy
46mConquering Distributed Cloud Logging Without Breaking the Bank
40mObservability that scales: Analyze data in stream and query remotely
40mTaming Spiky Log Volumes: Maintaining Real-Time Log Accessibility with Kaldb
46mBuilding a scalable logging platform