A Commerce-centric Approach to Queuing Fairly at High Throughput
summary
Claims a move from a stateless edge throttle to a stateful, commerce-aware queue that tracks ticket timestamps, inventory and fairness and runs as a Ruby gem in the application tier. Uses Nginx OpenResty Lua edge limiter initially, then Rails monolith + Redis Lua scripts to implement bined tickets and a working-bin scheduler. Rollout uses simulators and Genghis load tests; result is millions of pull requests per minute with p99 latencies of a few milliseconds and thousands of users completing checkout per second.
AI-written summary · about
chapters (12)
- 0:00The problem with stateless
- 2:12Flash sales traffic
- 4:08Shopify's traffic stack
- 6:29Implementing back pressure
- 9:12Traditional throttling methods
- 11:52Windowed approaches
- 17:37Legacy edge throttle
- 20:56Designing a fair system
- 25:00Stateful queue architecture
- 30:58Scheduler implementation
- 35:05Simulating and testing
- 37:47Results and conclusion
related talks
19mFlash Sale Engineering
47mFrom Legacy to High-Load: Evolution of a Web Application
51mDealing with eventual consistency
58mWelcome to the (state) machine
54mWhere we're going... we don't need batch jobs
10mLightning Talk: Resource Fairness and Utilization for Heterogeneous Batch/M...
22mOptimization strategies for a performant, headless storefront
24mQA