Making sense of Crypto Market Data in real time
summary
Bitvavo streams order flow from a low‑latency JVM core into Apache Kafka and ingests into Druid to compute candlestick aggregates for ~400 assets; the engine sustains 10k normal and bursts to 30k+ requests/sec with P99 latency around 5 microseconds and UI candlestick freshness at about 300–500 ms. Moving from DB cron jobs to streaming drops DB load ~40%, removes ~5,000 projection tables, cuts operational cost from €35k to €15k/month, and produces predictable, scalable real‑time candles.
AI-written summary · about
chapters (9)
- 0:00Bitvavo and crypto data
- 12:32Startup to scalable exchange
- 16:26Exchange engine performance
- 24:29Real-time candlestick challenges
- 28:00Kafka and Druid for market data
- 36:43Results and future plans
- 38:26Hiring and expansion
- 41:20Deployment and culture
- 51:24Q&A
related talks
20mAnalyzing streaming data with Apache Druid
28mBest of Stream + Batch: When Streaming Analytics Isn’t
21mPutting big data self-service analytics in the hands of business people
28mTransforming the Financial Services Industry with Event Stream Data
19mStock data analysis with
50mThe Real-Time Analytics Stack
29mDrilling for Data: Powering DeFi Applications
28mMoving Towards Real-Time OLAP - With First Principle Thinking and Guided by Access Patterns