Monolith to Microserv. at 20M+ requests per second: Latency and Scale Challenges

on YouTube ↗47m

" summary

RTB House migrates a bidding monolith into gRPC microservices handling 20M+ RPS, extracts geo, UA and URL parsers into independent services, and measures a 7 ms business-visible latency budget. Client-side DNS-based service discovery replaces 300+ load balancers, a custom batching layer (10-request windows) cuts required service instances ~3x, and swapping G1 for ZGC reduces P99 tail latency while increasing memory. Hedging with throttling and health probes preserves resiliency.

AI-written summary · about

" chapters (20)

1. 0:00Introduction
2. 1:12Migration overview
3. 3:00Real-time bidding at scale
4. 6:14Monolith challenges
5. 8:21Why split the monolith
6. 10:48Migration requirements
7. 13:40Microservices design
8. 15:45Practical challenges
9. 16:07Load balancing issues
10. 17:33Client-side load balancing
11. 20:50gRPC call overhead
12. 22:59gRPC streaming switch
13. 24:21Batching requests
14. 25:48Latency trade-offs
15. 27:13Garbage collection impact
16. 27:35ZGC for tail latency
17. 29:04System resilience
18. 30:09Hedging requests
19. 33:03Final architecture
20. 35:10Q&A

" related talks

• 35mLegacy Monolith to MicroservicesSmit Thakkar · DevConf.CZ 2019
• 48mMicroMonolithMichal Franc · Codemotion Amsterdam 2017
• 43mSolving Scalability by Converting a Monolith into MicroservicesArtem Nikulchenko · Highload fwdays'19
• 42mQ&A - Monolith to MicroservicesSam Newman, Sven Johann · GOTO Oslo 2020
• 51mAvoiding Microservice MegadisastersJimmy Bogard · CodeMash 2018
• 21mReal World Migration from HTTP to gRPC in RubyNao Minami · gRPC Conf 2020
• 22mA Case Study of Enhancing gRPC Micro-Services for Supporting High Throughput, Low Latency TradingHolly Casaletto, Yucong Sun · gRPConf 2024
• 53mHow do we build high‑load microservicesAnton Moldovan · .NET Fest 2017