How We Build Firebolt
summary
Firebolt hard‑forks ClickHouse as its runtime, stores data in a custom Triple F on S3 file format and uses a multi‑tier buffer manager that caches 10,000‑row index granules (sparse indexes loaded to local SSDs) so queries prune and fetch tiny ranges instead of whole files. Engines run as 1–128 node clusters, distributed execution uses a low‑latency shuffle and local SSD spilling, optimizer is an embedded fork of HyRise, metadata lives on FoundationDB, and SimilarWeb runs ~200 TB successfully on the system.
AI-written summary · about
chapters (18)
- 0:00Introduction
- 2:07Cloud Data Platforms
- 3:13Data Warehouses
- 7:18Real World Example
- 8:02Firebolt Overview
- 10:04High Performance Engine
- 12:02Clickhouse
- 13:10Merge Tree
- 20:59Sleep Mode
- 28:31Metadata
- 34:26Query Planner
- 36:19Highrise
- 39:30Distributed Processing
- 42:00Example
- 49:18Service Layer Orchestration
- 50:44Ecosystem
- 52:00Lessons Learned
- 53:20Questions
related talks
15m100x faster data app experiences delivered cost-effectively
01h 08mFirebolt: Why Powering User Facing Applications on Iceberg is Hard
04h 15mTopical Edition on Databases - Track 2
28mColumnstore Revolution
22mSolving Everyday Data Problems with FoundationDB
45mClickHouse: what is behind the fastest columnar database
06mLightning Talk: Performance Testing of FoundationDB using TPC-C
26mRunning FDB at scale at Wavefront