How Corning built E2E ML on a data lakehouse platform
summary
Corning centralizes images from global plants to S3 via AWS DataSync and trains models on the Databricks Lakehouse using Delta Lake, a Feature Store, and MLflow; a custom training loop produces a 200,000-parameter model (trained from scratch, >90% accuracy) exported for TensorFlow 1 edge inference, deployed via MLflow Registry and simple edge updates; tooling includes EKS labeling UI (CVAT API), Airflow orchestration, and DBX CI/CD; result: global rollout across plants and ~$2M cost avoidance in year one.
AI-written summary · about
chapters (3)
related talks
54mThe Future of Data Science/ML at Scale: A Look at MLflow, Delta Lake + Lakehouse
29mML in Production on Databricks
52mElevate your data and AI governance with Databricks Data Intelligence Platform
12mUsing AI-enabled Edge Cloud Technologies to Strengthen Europe’s...
28mBuilding Developer-Centric ML Inference Platforms
49mGenAI for Business on the Modern Data Stack
53mEnd-to-End ML
29mA Scalable Platform for Training and Inference Using Kubeflow