Scaling a monolith isn't scaling microservices

on YouTube ↗30m

" summary

Claim: the techniques that scale a large Rails monolith are not the same as those that scale microservices. Describes concrete tradeoffs between Resque and Sidekiq (Ruby task queues), message-oriented middleware / AMQP for service-to-service JSON messages, and Kafka as an event log. Gives numbers: Shopify queues tens of thousands of jobs a second and deploys core code ~60 deploys a day. Shows a working Kafka-powered dashboard that visualises >$1B in Black Friday/Cyber Monday sales.

AI-written summary · about

" chapters (10)

1. 0:00Start
2. 2:29Outline
3. 9:45Long running jobs - Resque
4. 10:55Large collections
5. 11:18Interruptible job with automatic resuming
6. 13:57Background jobs
7. 20:01Keep breaking changes manageable
8. 22:11Event-carried state transfer requires order information
9. 22:50Event notifications are commutative
10. 24:55Message oriented middleware

" related talks

• 19mMultigrain ServicesRod Cope · apidays London 2019
• 35mMicroservices: A Tale of 3 HeroesSha Ma, Jovel Crisostomo · Craft Conference 2021
• 38mMonolith vs. MicroservicesRafa Paradela · Scala Days 2023 Seattle
• 35mLegacy Monolith to MicroservicesSmit Thakkar · DevConf.CZ 2019
• 47mMicroservices? Monoliths? An Annoying Discussion!Eberhard Wolff · JCON-ONLINE 2021
• 29mWhen Web Scale Doesn't Make SenseJack Christensen · WindyCityRails 2016
• 41mDeconstructing the MonolithKirsten Westeinde · Craft Conference 2022
• 38mThe Dark Side of ScaleDeepak Giridharagopal · devopsdays Portland 2019