Evolution and Scaling of Feature Store at Uber
Talkat Open Source Summit Europe 2023 · on Tuesday, 2023-09-19· at 16:35· Room 0D-2-0D-3 (Floor 0), Abandoibarra Etorb., 4, 48011 Bilbao, Bizkaia, Spain·
summary
Palette runs 21,000+ features at 25M QPS with 99.999% uptime and shows exactly how that scale is achieved: ingest via Kafka, storage in Hive (offline) and Cassandra (online), and processing with Spark plus Flink for streaming. Users onboard features by submitting a JSON config; the platform applies auto-config, batching, incremental materialization and resume checkpoints to avoid full recompute. Delta and selective dispersals plus local/Redis/Cassandra caching and a metadata tiering system handle cost, freshness and online/offline consistency; embedding support (Sia) is in progress.
AI-written summary · about
related talks
33mEvolution and Scaling of Feature Store at Uber
30mHow Feature Stores Enable Operational ML
15mFeature Stores: Powering Machine Learning Infrastructure with Apache Flink
25mAccelerating ML at Uber with the Palette Feature Store
25mBuilding A Feature Store For Hyper Growth
32mA feature Store for Machine Learning
33mFeathr, an Open Source, Battletested, Scalable and Enterprise-Grade Feature Store
08mFeature Store Usage Patterns - From a Single Data Scientist to an Enterprise