Nexus Feature Store powering Disney Magic
summary
An internal system named Nexus provides a dual feature store: a row‑oriented low‑latency store and a columnar offline store used for batch training and inference, powering Disney+, Hulu, Star+, and ESPN+. Implementation uses Delta Lake on S3 for offline tables, a Postgres feature registry, and Redis (ElastiCache) for millisecond online reads with Python and Scala SDKs. A custom distribution engine serves single‑node Docker training jobs, supports regioned datasets, time‑travel, feature logging, and lifecycle management.
AI-written summary · about
chapters (7)
- 0:00Intro
- 1:11ML Platform overview
- 3:32Why do we need a Feature Store?
- 9:26Nexus Feature Store capability map
- 14:46Nexus FS Architecture - Components
- 15:49Nexus FS Architecture - Spark write & Non-Spark read
- 18:02Nexus FS Architecture - Streaming write & real-time read
related talks
30mHow Feature Stores Enable Operational ML
15mFeature Stores: Powering Machine Learning Infrastructure with Apache Flink
33mEvolution and Scaling of Feature Store at Uber
32mA feature Store for Machine Learning
30mAmazon SageMaker Feature Store: Store, discover, & share features for ML apps
33mFeathr, an Open Source, Battletested, Scalable and Enterprise-Grade Feature Store
20mNavigating the Skies of Data: A Flight Through Feature Stores
19mML feature store a cornerstone in modern data architecture for AI/ML