OpenMLDB: An Open-Source Real-Time Feature Platform
summary
OpenMLDB packages a Spark-improved batch SQL engine, a realtime SQL engine built from scratch, and a unified execution plan generator so the same SQL feature definition runs offline and online without rewrite. The realtime engine uses an in-memory double-layer skip list and pre-aggregation to hit millisecond latencies; deployments report about 4 ms for 1B orders/day and over 100 production applications including large Chinese tech firms.
AI-written summary · about
chapters (7)
- 0:00Introduction to OpenMLDB
- 1:49Why real-time features matter
- 3:55Challenges in feature engineering
- 6:26OpenMLDB architecture overview
- 8:37Key features and optimizations
- 10:50Enterprise-grade capabilities
- 11:33Real-world use case
related talks
26mMLbase: A Distributed Machine Learning System
14mReal-Time Data Processing for ML Feature Engineering
26mOracle Machine Learning: what's new and what's next?
28mCharge Your Database with Predictive ML Capabilities using Open-Source
32mLiveMLP: ML Platform for Assisting Contact Center Agents in Real-Time
32mOpenTelemetry, databases, and machine learning
29mML in Production on Databricks
20mEnabling Machine Learning through Real-Time Data Processing