I Can’t Believe It’s Not A Queue: Using Kafka with Rails
summary
Explains concrete claims and demos: Kafka handles hundreds of thousands to millions of messages a second, Heroku launches a public beta Kafka add‑on with SSL and certs, and Ruby apps use ruby‑kafka or jruby‑kafka (tradeoffs shown). Demo ingests Heroku log drains into Kafka with an async producer, uses consumer groups to compute metrics in Redis, shows default topic creation with 32 partitions and at‑least‑once delivery, and replays production traffic to staging; code and Docker setup live on GitHub.
AI-written summary · about
chapters (7)
- 0:00Start
- 3:03What is Kafka?
- 3:14Kafka is a distributed, partitioned, replicated commit log service. It provides the functionality of a messaging system, but with a unique design.
- 3:54Fast Scalable Durable
- 4:29SKYLIGHT
- 17:42Build metrics based on web traffic
- 30:10Heroku API Event Bus
related talks
42mSimplifying Logs, Events and Streams: Kafka + Rails
35mSo You’ve Got Yourself a Kafka: Event-Powered Rails Services
31mEvent Streaming on Rails
38mEffective Data Synchronization between Rails Microservices
30mRuby in the Billions
27mRails Orbiting the cloud: Mastering the scalability with Kubernetes
27mScaling a Rails Monolith w/Event Driven Microservices
28mReal-Time Rails