Fast, Faster, Fastest: Object Storage, Cloud Block Storage, and SSD
summary
Claim: for ClickHouse workloads storage often isn’t the bottleneck — local NVMe, EBS and S3 give similar query latency because ClickHouse does heavy compression, parallel I/O and its own caching. Tests use key_ioer, fio and click-bench and show non‑linear VM storage scaling, EBS sometimes 30–39% faster on certain instance types, and object storage slower for tiny queries but comparable for bulk reads. Note operational caveat: bootstrapping can be 10x–100x slower on EBS for massive scans.
AI-written summary · about
chapters (26)
- 0:00Introduction
- 1:08Why storage matters for analytical databases
- 1:53Local NVMe SSDs in cloud
- 3:38Introducing KPerf storage benchmark
- 6:26Cloud block storage with EBS
- 7:52Object storage S3 characteristics
- 9:57Comparing NVMe EBS and S3
- 10:18ClickHouse architecture and Kubernetes
- 12:27Altin ClickHouse Operator
- 13:08Drawbacks of local NVMe in Kubernetes
- 14:53Testing storage impact with ClickBench
- 15:56ClickHouse cached vs direct I/O
- 17:20EBS vs local NVMe performance
- 18:26ClickHouse internal caching
- 19:27Why EBS outperformed local NVMe
- 20:33S3 storage performance with ClickHouse
- 21:15When storage affects performance
- 22:22ClickHouse benchmark insights
- 23:30Storage bottlenecks in restarts
- 24:15Storage not always bottleneck
- 24:57Tuning EBS for high performance
- 25:38Databases optimizing I/O
- 26:01Importance of caching
- 27:04Limitations of generic benchmarks
- 27:46Optimizing schemas and queries
- 28:51Tools for ClickHouse
related talks
29mAccelerate your cloud architectures with the right storage options
42mClickHouse: what is behind the fastest columnar database
30mBuilding For Fast
28mColumnstore Revolution
34mClickHouse® - What is Behind the Fastest Columnar Database?
37mCloud Storage 101
42mTips & Tricks to Keep Your Queries under 100ms with ClickHouse®
41mThe New Generation Of Data Stores