Cortex: Ingesting and Querying 100s of Billions of Samples Per Day
summary
Introduces Prometheus and then explains Cortex, a horizontally scalable distributed implementation of Prometheus for ingesting and querying hundreds of billions of samples per day. Covers Cortex’s architecture—including decoupled, independently scalable ingest and query paths—traces the original design, the bottlenecks encountered, subsequent architectural changes, and current community work and future directions.
AI-written summary · about
chapters (17)
- 0:00Start
- 4:25The Problem
- 6:22Solution: Project Frankenstein
- 7:25Frankenstein - Cortex
- 10:27Different ingest and query path
- 10:54Datamodel Prometheus
- 11:30Better indexing 11 schemas!
- 14:28Index Caching
- 15:04Query Caching
- 15:58Query Parallelisation
- 18:55Getting Started
- 20:02You're done!
- 20:18#2 Simple Setup
- 20:52Easy to operate?
- 21:34Status tomorrow
- 21:58TSDB inside Cortex
- 23:15Store: make index cache shared cache triendly #1825
related talks
22mCortex: Ingesting and Querying 100s of Billions of Samples Per Day
37mIntro: Cortex
28mCortex 101: Horizontally Scalable Long Term Storage for Prometheus
45mScaling Prometheus with Cortex, one time series at a time
39mDeep Dive: Cortex
20mCortex: Intro and Production Tips
16mCortex: Multi-tenant Scalable Prometheus
20mIntro to Scaling Prometheus with Cortex