Lend Me Your Scala, I'll Trade You Some Data
summary
A backend and data engineer present rebuilding a "born into store" feature that detects physical store visits from campaign data using Kafka and Spark pipelines, HTTP endpoints, and Google Cloud services. They explain rewriting the system from scratch to add metrics, logs, clearer documentation, and better partner endpoints, which reduced lost messages, clarified ownership, and improved on-call and handovers; the feature now covers 35% of campaigns.
AI-written summary · about
chapters (11)
- 0:00Introduction
- 2:11Expert pressure consequences
- 3:59Store feature challenges
- 5:47Collaboration and humility
- 8:18Scala in distributed systems
- 10:44Scala developer perception
- 13:32Sustainable solutions partnership
- 15:22Asking for help and limits
- 16:48Written and async communication
- 19:23Definition of done
- 21:13Feedback and next steps
related talks
42mBuilding a Company on Scala
38mMy first year in Scala
33mData: The Land DevOps Forgot
51mScala At Netflix
51mDeus Ex Scala: Finding a Home for Scala in the Corporate Machine
42mScaling Data at Ovoko: The Succesful Strategy
48mBuying into Scala from the Inside and Out
33mAccidental DataOps: How we matured our organization with data, and how you can too!