A feature Store for Machine Learning
summary
Feature engineering is productized: Hopsworks.AI provides a versioned DataFrame API and Apache Spark jobs that materialize feature groups into a dual online/offline store so training joins and low‑latency serving use the same values. The platform supports time‑range queries via a Hoodie storage layer, JDBC online lookups for prepared feature vectors, connectors for S3/GCS/Azure, and a cloud managed hopsworks.ai offering; an example customer handles ~40 TB of transaction data.
AI-written summary · about
chapters (15)
- 0:00Introduction
- 1:31Data science gap in feature engineering
- 4:18What is a feature store
- 7:26Feature types and update frequencies
- 8:50Dual database architecture
- 9:55Data frame API for ingestion
- 11:19Creating feature groups
- 13:07Training datasets for ML
- 15:51Versioning features and datasets
- 17:57Time travel and incremental updates
- 19:43Online feature store for real-time
- 21:49End-to-end ML pipeline
- 23:56Demo of Hopsworks
- 26:43GDPR compliance and isolation
- 28:51Questions and answers session
related talks
23mHopsworks Feature Store after 4 years: Lessons learned and what's next
15mKick-off
30mHow Feature Stores Enable Operational ML
15mFeature Stores: Powering Machine Learning Infrastructure with Apache Flink
30mAmazon SageMaker Feature Store: Store, discover, & share features for ML apps
08mFeature Store Usage Patterns - From a Single Data Scientist to an Enterprise
33mFeathr, an Open Source, Battletested, Scalable and Enterprise-Grade Feature Store
20mNavigating the Skies of Data: A Flight Through Feature Stores