DataEngBytes 2023 - SYD-T1-10 - Kanishka Mohaia
summary
Rokt runs a web‑scale real‑time ML stack that hits 450 ms P99 latency while serving about 17M requests/day across roughly 60 concurrent models and terabytes of data. Architecture evolves from Lambda/EC2 to self‑hosted Kafka and Kubernetes, using Kubeflow + TFX + TF‑Serving for training and serving; a custom feature store is built because vendor options miss latency needs. Cost and drift drive choices; deployment is automated with A/B pipelines and ongoing work on automated feature selection.
AI-written summary · about
chapters (5)
- 0:00Introduction and context
- 2:16Challenges at web scale
- 8:09Design considerations
- 14:06The evolution of Rockt's stack
- 20:38ML pipelines and future steps
related talks
39mDataEngBytes 2023 - BRI-T1-02 - Mikiko Bazeley
29mMachine learning in production
41mBig Data Serving: The Last Frontier. Processing and Inference at Scale in Real Time
36mAccelerating High-Performance Machine Learning at Scale i...
42mDeploying ML at Scale with Kubernetes and TFX
53mData Engineering and AI
30mDataEngBytes 2023 - MEL-T2-01 - Mikiko Bazeley
42mThe Challenges Managing a Kubernetes-Based Machine Learning Infrastructure