How to Build a State Machine on MongoDB and Redis
summary
CRM flow engine Canvas pairs MongoDB for single-write user state with Redis for coordination and caching, using Sidekiq/sidekick jobs. Key techniques: a Redis lock key: journeyID+userID+entryTimestamp to guarantee single-branch progression, an extra read-before-lock to cut a ~20–30% wrong-lock rate, and batching users into Redis sets with dispatch jobs pulling ~200 users to reduce millions of per-branch jobs to orders of magnitude fewer (≈thousands). The system commits one MongoDB write at send time and uses Redis-held branch IDs for debugging.
AI-written summary · about
chapters (10)
- 0:00State machine on MongoDB and Redis
- 1:24Company background
- 2:46Canvas state machine motivation
- 5:15User state tracking
- 9:33Branching challenges
- 13:06Redis locks for transitions
- 16:57Optimizing lock checks
- 17:19Batching jobs for loads
- 20:08Redis sets for dispatching
- 22:14Q&A
related talks
56mRedis Application Patterns in Rails
46mFun with Ruby and Redis
34mScaling Redis writes with cluster mode
44mRunning atomic transactions across multiple data models?!?
27mBehind the Keys: Redis Oyster Cult
35mMongoMapper, Mixins, and Migrations - a look at managing data-model complexity
59mUsing Redis beyond caching
36mBuild a Cloud Native Asynchronous Messaging System For Scale With Redis