Cache for Cash—Speeding Up Production with Kafka and MySQL binlog
summary
Company pushes MySQL changes into Kafka and MySQL binlog to update front-end caches instead of pulling, turning millions of background TTL refreshes into a push stream. System hits ~500K HTTP RPS, one machine sees >1M cache reads/sec and ~13K cache updates/sec with minute TTLs; push path delivers changes globally in ~2–3 seconds, lets services use embedded Kafka consumers to inject cache entries, cuts redundant updates by an order of magnitude and reduces CPU/load spikes (observed 10× update count vs 30× load difference) while preserving consistency.
AI-written summary · about
chapters (8)
- 0:00Intro
- 0:14Cache for Cash Speeding up production with Kafka and Mysql binlog
- 5:013 Constraints of a Successful Production
- 8:14Traditional cache layers
- 8:28Fresh Data = higher yield
- 11:49Lower load = Lower cost
- 19:36Use Kafka for data distribution
- 21:09Inject Data to Caches
related talks
32mA Change-Data-Capture use-case: designing an evergreen cache
33mCache is King: Get the Most Bang for Your Buck From Ruby
18m"Kafka's Metamorphosis"
32mA Change-Data-Capture use-case: designing an evergreen cache
41mIntegrating Kafka with a Real-Time Database
24mMigrations under Production Load: How to Switch...
53mDemystifying Kafka
16mKVCache Cache in the Wild: Characterizing and Optimizing KVCache Cache at a...