Whisky Clustering with Apache Projects: Groovy, Commons Math, Ignite, Spark, Wayang, Beam and Flink

on YouTube ↗34m

" summary

Uses an 86 distilleries dataset of whiskey flavor scores and runs plain and distributed K‑means implemented with Groovy plus Commons Math, then distributed via Ignite, Spark, Wayang, Beam and Flink. The pipeline produces centroids and medoids, reduces 12 flavor dimensions with PCA + elbow method for cluster count, and exposes cluster assignments; code and notebooks run from GitHub/Gitpod. Also demonstrates online K‑means for streaming updates; distributed approach shares centroids between nodes (works but needs sensible partitioning and serialization).

AI-written summary · about

" chapters (13)

1. 0:00Introduction and overview
2. 3:39Groovy for data science
3. 7:06What is clustering
4. 13:03Whisky data and features
5. 17:59Visualizing clusters
6. 21:07Scaling clustering algorithms
7. 23:12Apache Ignite for K-means
8. 25:18Spark distributed K-means
9. 25:59Wayang unified ML framework
10. 28:47ML for Apache Beam
11. 30:14Beam K-means implementation
12. 31:58Flink and streaming K-means
13. 33:21Q&A and closing

" related talks

• 36mIn the Name of WhiskeyJulia Ferraioli · Ruby on Ales 2016
• 48mHow to make intelligent web-appsDeepak Thukral · EuroPython 2011
• 51mApache Spark and machine learning: dreams and realitiesNastasia Saby · ScalaIO 2019
• 18mCluster Serving: Distributed Model Inference using Apache Flink in Analytics ZooJiaming Song, Dongjie Shi · Flink Forward Global Virtual 2020
• 32mBigData Clustering Illustrated by OK GiftsArtur Kadurin · HighLoad++ 2014
• 30mAlice and Mad Hatter....Marianna Diachuk, Roksolana Diachuk · Scale By the Bay 2021
• 01h 25mApache Spark for Big Data ProcessingLudwine Probst, Ilayaperumal Gopinathan · SpringOne2GX 2015
• 18mCluster Serving: Distributed Model Inference using Big Data StreamingLei Xia, Jiaming Song, Dongjie Shi, Qiyuan Gong, Jinquan Dai · OpML '20