From Service To Platform: A Ranking System in Go
summary
Builds a modular ranking platform called pipedream in Go: an acyclic graph of small stages, plus series/parallel meta-stages and middleware. Uses goroutines with errgroup, copy-the-request to avoid races, injectable constructors, and a dynamic config feature-flag to disable bad stages. Claims this structure cuts accidental complexity, replaces ad hoc profilers with per-stage latency metrics, and scales from a single livestream service to multiple Reddit product surfaces while letting other teams add pipelines without breaking existing code.
AI-written summary · about
chapters (8)
- 0:00Intro
- 6:33Recommendation System
- 12:30Ranking Pipeline
- 19:00Exporting Types
- 21:35Principles
- 23:45Interfaces
- 24:55Middleware
- 27:03Conclusion
related talks
42mFrom Service to Platform- A Ranking System in Go
28mManaging and Scaling Real Time Data Pipelines using Go
46mSimplifying Scalable Cloud Application Development with Go
37m"Using Golang for Platform Engineering"
49mFrom C# to Golang, My Data Science Journey
01h 00mConsistency Catalyst: The story of Paddle's in house microservice toolkit
15mI came for the easy concurrency I stayed for the easy composition
45mGo, Microservices and all the joy!