TrailDB tutorial Store and process billions of events efficiently
summary
TrailDB claims an immutable C library with Python bindings that stores billions of events and compresses datasets tightly — e.g. a Wikipedia dump of ~600 million edits becomes 5.8 GB for 600M edits (~9 bytes/event). It powers production use (AdRoll reports 20 trillion events at AdRoll), supports S3-backed immutable files, lets single servers hold and query archives, and pairs with tools like DataShader for large-scale visualizations; the library and file-based workflow actually work in production.
AI-written summary · about
chapters (11)
- 0:00TrailDB tutorial intro
- 2:11Why TrailDB was created
- 18:48Developer trends
- 29:09How TrailDB compresses data
- 33:39Installing TrailDB
- 40:34First TrailDB file
- 51:25Querying in Python
- 1:06:32Filtering events
- 1:10:09Wikipedia edits visualization
- 1:17:12Performance tips
- 1:20:19TrailDB in production
related talks
05mDBTB INT Slava Akhmechet
25mProcessing Biggish Data with DuckDB and Python
41mEnhancing Event Analysis at Scale: Leveraging Tracking Data in Sports
22mIt’s all about the db - a journey to accommodate 100k events per second
36mHow Can HTAP Databases Simplify Your Data Infrastructure?
24mGive meaning to 100 billion analytics events a day, analytics at Teads
48mHow to Analyze Terabytes of Data from GitHub Archive at High Speed
45mAnalytics for not so big data with DuckDB