Empowering your feature stores with AI feature discovery
summary
Tool from dotData automatically generates features on Databricks using Delta Lake and Spark, exploring about 100,000 feature hypotheses and returning a selected set (example: 49 features chosen) as a flat DataFrame that can be pushed to Databricks Feature Store. Configuration uses data slots, temporal join windows (six months in the demo), and neural/weekly encodings; a Python package (DDPi, not yet pip-installable) runs the discovery, selection, and materialization pipeline end to end.
AI-written summary · about
chapters (6)
- 0:00Introduction to AI feature discovery
- 1:26Example use case setup
- 3:50Code implementation walkthrough
- 6:18Feature selection results
- 8:03Generating feature values
- 9:28Workflow and conclusion
related talks
20mNavigating the Skies of Data: A Flight Through Feature Stores
52mElevate your data and AI governance with Databricks Data Intelligence Platform
32mA feature Store for Machine Learning
28mData VS Engineers: a long lasting story
49mBuild & deploy generative AI apps in days with Databricks Lakehouse AI
15mFeature Stores: Powering Machine Learning Infrastructure with Apache Flink
30mAmazon SageMaker Feature Store: Store, discover, & share features for ML apps
30mHow Feature Stores Enable Operational ML