How Spotify Leverages Flyte To Coordinate Financial Analytics Company-Wide
Talkat KubeCon + CloudNativeCon North America 2021 · on Friday, 2021-10-15· at 14:30· 411 Theater + Online·
summary
Spotify standardizes its financial forecasting pipeline "One Model" on Flyte to replace a messy, eight-team quarterly process that used to take 3–4 weeks with automated, versioned workflows that run in hours and enable ad hoc scenario analysis. Flyte enforces typed tasks, offers caching avoids reruns across teams, runs Python/Java/Scala tasks (Spark plugin for big jobs), records immutable versions, and exposes a UI and CLI for launches and monitoring — data teams report faster feedback, parallelism, and fewer handoffs.
AI-written summary · about
chapters (14)
- 0:00Intro
- 0:50What is Financial Forecasting
- 5:12Requirements Gathering Phase
- 5:39Problem Statement
- 9:45Key Differences
- 15:12Assumptions
- 18:20Promote successful workflows
- 22:14Workflows
- 29:45UI Walkthrough
- 31:32Graphs
- 34:04Why Flyte
- 35:04Flyte Workflows
- 38:00Flink
- 38:31Unified Dev Experience
related talks
28mRun Your Data and Machine Learning Workflows on Kubernetes with Flyte
38mFlyte: Cloud Native Machine Learning & Data Processing Platform
23mScaling the Sound: Fleet Management at Spotify
38mBuilding and Managing a Centralized Kubeflow Platform at Spotify
34mRunning Large-Scale Stateful Workloads On Kubernetes at Lyft
30mSpotify Harnesses Ray for Next-Gen AI Infrastructure
19mHigh Performance Backstage for High Performing Organizations
40mA study of a large-scale micro component architecture