Introducing Zipline: An Open Source Feature Engineering Platform
summary
Chronon is an open source feature engineering platform (rebranded from Zipline) that claims point‑in‑time correctness, online/offline consistency, and millisecond online fetches. It compiles Python configs into Spark jobs to generate backfills, streaming refresh, and quality metrics, supports CDC and snapshots for dim tables, and plugs into Spark + Hive + Airflow + KV store for real‑time serving. A 5‑minute demo shows developing real‑time features and the API’s group‑by/time‑window primitives.
AI-written summary · about
chapters (14)
- 0:00Introduction and setup
- 1:10What is a feature platform
- 4:42Feature platform goals and pillars
- 6:57Zipline API design principles
- 10:14Computation models and goals
- 12:03Non-goals of Zipline
- 13:07System requirements
- 14:11Example problem statement
- 17:24E-commerce feature example
- 18:49Data sources overview
- 21:20Event vs. snapshot sources
- 22:42Point-in-time accuracy
- 23:48Snapshot and mutation handling
- 24:51Architecture overview
related talks
34mZipline - A Declarative Feature Engineering Library
36mReal-Time ML Pipelines: Feature Chaining with Chronon
23mHow we unified feature engineering across data and backend
01h 28mFeature Engineering Made Simple
36mDeclarative Feature Engineering
31mSpark challenges
45mBuilding a Feature Platform to Scale Machine Learning
41mApache Zeppelin, the missing GUI for your Big Data backend