Building A Feature Store For Hyper Growth
summary
Doordash builds an internal declarative framework fabricator that makes a feature one-line deploy and available to models within hours, which triggers hypergrowth in models and features. Online store starts on Redis (Elasticache), requires upscales every two weeks and cluster resizes that can take 12–24 hours, so team prototypes a complementary disk-backed store using cockroachdb (CRDB). CRDB rollout takes ~nine months; rekeying and batched uploads plus grouping features by source cut p99 latency ~50% and upload compute ~60–70%, trading write complexity for lower eviction risk and cheaper storage.
AI-written summary · about
chapters (6)
- 0:00Intro
- 0:44What drove our growth
- 4:41What is fabricator
- 7:42Journey to CRV
- 14:26Cocker DB
- 19:57Optimizations
related talks
30mHow Feature Stores Enable Operational ML
33mEvolution and Scaling of Feature Store at Uber
44mFeature store
38mScaling Data Science
19mML feature store a cornerstone in modern data architecture for AI/ML
29mIntegrating High Performance Feature Stores with KServe Model Serving
30mDataEngBytes 2023 - MEL-T2-01 - Mikiko Bazeley
32mA feature Store for Machine Learning