OpenTSDB - Time Series Schema on Schemaless NoSQL
summary
OpenTSDB history and schema decisions explained with concrete tradeoffs: a production HBase deployment handling about 10 million writes per second (peaks ~18M) and ~3PB uncompressed (500TB compressed), storing metrics as HBase row-key: salt|metric|base_ts|sorted tags and encoding points with VLE for integers/floats; UID tables reduce string storage but create lookup and cardinality limits; rollups, in-memory gorilla-style caching, Kafka+Storm for streaming and alerts, and a 3.0 rewrite for pluggable storage and distributed queries are shown with measured benefits and operational burdens.
AI-written summary · about
chapters (32)
- 0:00Who am I
- 0:52Time Series
- 1:12Graph Time Series
- 1:33Data Points
- 2:17Data Types
- 2:48Flat Files
- 3:33Pain to Manage
- 4:08RDAM
- 4:43Schemas
- 5:25Sharding
- 5:44Storing Procedures
- 6:58UUID System
- 7:17Strings
- 7:40Common Problems
- 10:30Mrtg
- 12:28Time Series Data bases
- 13:08Graphite
- 15:28HBase BigTable
- 21:46Salesforce
- 22:27Yahoo
- 23:30Benefits
- 25:29Coming up in 30
- 27:30Druid
- 29:48FluxDB
- 31:43PrometheusDB
- 32:16DMS at Yahoo
- 33:06Thank you
- 33:46HBase Cluster
- 35:12Tags
- 37:01Models
- 40:19Monitoring Pipeline
- 42:15Conclusion
related talks
07mTime Series Databases
30mCatching Waves with Time-Series Data
29mCatching Waves with Time-Series Data
39mTimescaleDB: Re-engineering PostgreSQL as a time-series database
39mA New Data Model for Time Series Data Processing
01h 06mReconstructing History with XTDB
49mIoTDB - Moving from Relational to Timeseries- 39mA New Data Model for Time Series Data Processing