Accelerating QuestDB: Lessons from a 6x Performance Boost
summary
QuestDB (Apache‑2.0) moves from being 33× slower on ClickBench to roughly 3× and reports up to a 6x performance boost on some queries by adding SIMD-style string scans, a smjit JIT for compiled predicates, specialized hash tables and a 256-way parallel merge for high‑cardinality GROUP BYs, plus UTF‑8 “German” string storage and faster regex/replace paths. An improved as-of join tolerance and binary‑search join strategy cut expensive backward scans. All changes live in QuestDB’s GitHub and show measurable speedups.
AI-written summary · about
chapters (22)
- 0:00Introduction to QuestDB
- 1:03Why databases need speed
- 3:51QuestDB architecture and design
- 6:41Time series query optimizations
- 8:06Improving non-time queries
- 9:10Choosing the right benchmark
- 10:16ClickBench performance baseline
- 12:44Performance improvements over time
- 15:54Two-year performance journey
- 16:35Simple optimizations that worked
- 18:01Just-in-time compiler enhancements
- 20:50Parallelizing string filters
- 23:25Query rewriting techniques
- 23:46SIMD and low-level optimizations
- 25:34String storage improvements
- 26:39Handling variable-size data
- 29:50Optimizing group by operations
- 31:36High cardinality group by
- 32:58Specialized hash tables
- 34:22As-of join improvements
- 37:57Tolerance in as-of joins
- 38:40Lessons learned and future plans
related talks
01h 07mQuestDB: Fast Open Source Time Series Database
43mIngesting over 4 million rows a second on a single instance
52mBoosting Typical Query Patterns: PostgreSQL 18's Performance Enhancements
38mAccelerating Query Performance in CrateDB
50mImproving Postgres' Efficiency
30mPerformance: Tips & Tricks to Optimize the Speed of Your Mission-Critical Database
15mQuake: Adaptive Indexing for Vector Search
41mUsing Databases to Pull Your Application Weight