Stream Processing for Analysts with Flink and Nussknacker

on YouTube ↗35m

" summary

A working pipeline links Kafka ingestion to Flink processing and a graphical Nussknacker UI so analysts deploy stateful streaming logic without writing Scala; the system handles windowed aggregations, HyperLogLog cardinalities, model scoring via REST or PMML, and sandbox plus cloned integration environments for safe testing. Concrete claims: sub-second latency on event reaction, 200–300k expression evaluations/sec using Spring EL, and deployed fleets processing 150–200k events/sec per cluster with dashboards for per-job monitoring and rollback.

AI-written summary · about

" chapters (6)

1. 0:00Start
2. 0:59mobile operator
3. 3:50Eraud detection
4. 5:12tream operations
5. 19:38Expression Language
6. 22:44Enricher

" related talks

• 36mBuilding a Self-Serve Stream Processing Platform with FlinkPing Zou, Manuela Wei, Nick Hwang · Flink Forward Global Virtual 2020
• 37mBuilding and operating large-scale streaming applications at KingGyula Fora · Flink Forward Berlin 2017
• 50mPanel: Large Scale Streaming in ProductionMonal Daxini, Maxim Fateev, Ted Dunning, Xiaowei Jiang, Jamie Grier · Flink Forward Berlin 2016
• 40mGrowing a managed Flink streaming platform from scratchChen Qin · Flink Forward Global Virtual 2020
• 38mBuilding Unified Streaming Platform at UberZhenqiu Huang, Venkata Sanath Muppalla · Flink Forward Global Virtual 2020
• 46mKeep it going — How to reliably and efficiently operate Apache FlinkRobert Metzger · Flink Forward Berlin 2017
• 58mBuilding Self Service Platforms for Apache FlinkGorman, Nienhuis, Canzonieri, Novak · Flink Forward Europe 2019
• 46mWorkshop: Log Processing with Apache NiFi and Apache KafkaSophia Izokun · AI.dev: Open Source GenAI & ML Summit North America 2023