Flink Forward Berlin 2017
| title | author | length |
|---|---|---|
| Bring Flink to Large Scale Production | Xiaowei Jiang | 47m |
Alibaba runs Flink at 100 million events/sec peak using RocksDB state backend and introduces async external I/O to avoid blocking, yielding 2–3x throughput on… | ||
| Taking away customer friction through streaming analytics | Ferd Scheepers | 38m |
ING makes a concrete claim: a global streaming platform using Apache Flink plus targeted SaaS rules replaces brittle homegrown streaming and reduces customer… | ||
| data Artisans Product Announcement | Patrick Lucas, Robert Metzger | 37m |
Announces dA Platform 2 that packages Apache Flink with an Application Manager that treats jobs as declarative applications, tracks metadata and savepoints and… | ||
| NSDB (Natural Series DB) | Roberto Bentivoglio, Saverio Veltri | 28m |
Radicalbit builds NSDB as a time-series serving DB implemented in Scala that uses Akka cluster for distribution, Lucene-style indexing for space (dimensions)… | ||
| Large Scale Stream Processing with Blink SQL at Alibaba | Shaoxuan Wang | 27m |
Alibaba runs Blink SQL (Flink-based) as a production stream SQL engine after ~2 years of work, powering search, recommendation and ads and running most… | ||
| Drivetribe’s Kappa Architecture With Apache Flink® | Aris Koliopoulos | 32m |
Drivetribe builds a Kappa style event-sourced pipeline that treats Kafka as the source of truth and uses Apache Flink to compute real-time, personalized feed… | ||
| The State of Flink and how to adopt Stream Processing | Stephan Ewen | 45m |
Apache Flink powers large production pipelines and attendees report 2.1 trillion events/day processed by their Flink deployments; collaboration with Alibaba… | ||
| Do I Know You? Real time facial recognition with an Apache stack | Trevor Grant | 33m |
Full stack lambda-style system computes Mahout/Spark distributed SVD to produce eigenfaces (130‑d vectors) from a 10,000-face dataset in about 45 minutes, uses… | ||
| Custom, Complex Windows at Scale using Apache Flink | Matt Zimmer | 34m |
Netflix moves batch badge ETL into Apache Flink by building a custom Window API layer that creates unaligned, event‑type bounded windows, handles millions of… | ||
| Keep it going — How to reliably and efficiently operate Apache Flink | Robert Metzger | 46m |
A concrete capacity example shows 1 million msgs/s at 2 KB each feeding a five‑minute sliding window with 500 million unique keys, producing ~2 GB/s total and… | ||
| Using Queryable State to update ML Algorithms per datapoint for... | Vincent D Warmerdam, Fokko Driesprong | 43m |
A streaming matchmaking pipeline built on Apache Flink uses Queryable State to hold per-player Bayesian histograms and updates skill estimates per game event… | ||
| Parameter Server on Flink, an approach for model-parallel machine learning | D. Berecz, G. Hermann | 38m |
A Parameter Server implemented on Flink Streaming API stores model shards on server operators and runs updates on worker operators, supporting model-parallel… | ||
| From Flink 1.3 to 1.4 | Till Rohrmann | 38m |
Flink 1.3 adds concrete features: incremental checkpoints shrink checkpoint size (example production case 60 GB -> 1–30 GB) and cut checkpoint time (180s ->… | ||
| Deploying Flink Jobs as Docker Containers | Dominik Bruhn | 17m |
A concrete pipeline is described that builds Flink job jars into Docker containers, pushes images to a registry, and deploys them automatically onto a YARN… | ||
| Flink-JPMML: An open source Flink library for Streaming ML Model Serving | A. Spina, F. Frontera | 36m |
An open source Flink operator that serves PMML models for streaming predictions, loading models on demand from HDFS or a model repo via a broadcast control… | ||
| Complex Event Processing with Flink: the state of FlinkCEP | Kostas Kloudas | 33m |
Flink Forward Berlin, September 2017 #flinkforward
Kostas Kloudas, Software Engineer at data Artisans
Pattern matching over event streams is increasingly… | ||
| Extending Apache Flink stream processing with Apache Samoa ML methods | Piotr Wawrzyniak | 31m |
Integration connects Apache Flink to the stream ML engine Apache SAMOA via Kafka connectors so each labeled instance updates models online and predictions flow… | ||
| Building a network stack for optimal throughput / low-latency trade-offs | Nico Kruber | 23m |
Flink 1.3 removes brittle buffer config and uses 32KB memory segments, a global and per-subtask local buffer pool, and Netty/TCP for transport; work for 1.4… | ||
| “Hit me, baby, just one time” – Building End-to-End Exactly-Once Apps w/ Flink | Piotr Nowojski | 26m |
Explains how Flink uses checkpointing and a two-phase commit sink function to coordinate internal state, external transactions and checkpoint barriers so… | ||
| I²: Interactive Real-Time Visualization for Streaming Data with Flink & Zeppelin | Philipp Grulich, Jonas Traub | 22m |
I² connects Apache Flink and Zeppelin to drive live visualizations by sending UI changes back into the pipeline and adapting operators at runtime, using a… | ||
| Managing State in Apache Flink | Tzu-Li (Gordon) Tai | 29m |
Shows how Flink stores operator state as state snapshots to distributed filesystem using checkpoint barriers, and contrasts JVM-heap vs eager vs lazy… | ||
| Introducing FlinkML | Boris Lublinsky, Stavros Kontopoulos | 21m |
A Flink-based model-serving framework treats exported models as data, uses protobuf model-as-data to carry a binary model blob plus metadata, and stores large… | ||
| Cypher-based Graph Pattern Matching in Apache Flink | Max Kießling, Martin Junghanns | 41m |
A Cypher engine runs on Apache Flink by translating MATCH patterns into embeddings and Flink operators (leaf, join-embeddings, expand-embedding using iterative… | ||
| Large Scale User Behavior Analytics by Flink | Hao Wu | 43m |
A real-world UBA pipeline runs Flink as the streaming engine, handles 10k+ unique users across 10+ dimensions, and targets bursts near 100k TPS. CEP logic… | ||
| Building and operating large-scale streaming applications at King | Gyula Fora | 37m |
King runs Apache Flink jobs that consume over 40 billion events per day and maintain multi‑TB user state (publisher lists over 10 TB). A single Flink backend… | ||
| Moving Beyond Moving Bytes | Joey Frazee, Suneel Marthi | 39m |
Claim: streaming pipelines must use shared schemas to enable projection, join and aggregation rather than “moving bytes.” Demo wires Apache NiFi → Kafka →… | ||
| Unified stream and batch processing with Apache Flink’s relational APIs | Fabian Hueske | 43m |
Apache Flink provides Table API and SQL as a single relational layer that yields identical results on files or Kafka streams by translating queries into the… | ||
| The Approximate Filter, Join, and GroupBy | David Rodriguez | 45m |
Introduces a practical pipeline that turns numeric or string arrays into hash signatures so arrays that are “close enough” collide, using locality-sensitive… | ||
| Huawei Cloud Stream Service in practice | Dr. Radu Tudoran | 38m |
Beta cloud stream service uses Flink as execution engine and exposes a SQL client that compiles via Calcite, deploys containerized jobs through a REST API and… | ||
| Build a Real-time Stream Processing Pipeline with Apache Flink on AWS | Steffen Hausmann | 44m |
Demonstrates building a live AWS pipeline that connects Apache Flink, Amazon Kinesis, and EMR, with Elasticsearch for indexing and CloudWatch for metrics and… | ||
| Predictive Maintenance with Apache Flink | Dongwon Kim | 41m |
Describes a deployed Predictive Maintenance product called metatron PdM that runs a DNN trained with Keras on TensorFlow/Theano against live sensor streams… | ||
| Migration of a realtime stats product from Storm to Flink | Patrick Gunia | 28m |
ResearchGate migrates a realtime stats pipeline from Storm to Flink, replacing per-event +1 DB writes with time-window pre-aggregation and keyed state, cutting… | ||
| Flink in Containerland | Patrick Lucas | 33m |
Built and published an official Flink Docker image and runs Flink on Kubernetes, but networking and addressability cause the biggest failures: Akka-based RPC… | ||
| A Materialization Engine for Data Integration with Flink | Mihail Vieru | 21m |
A Flink-backed engine reads topic partitions from Nakata (Kafka), deploys one short-lived batch job per request via Flink’s REST API, and writes per-partition… | ||
| Flink-Kudu connector | Rubén Casado Tejedor | 22m |
Connector that reads/writes Kudu using Flink DataSet and DataStream APIs and implements a Kudu InputFormat, a RichOutputFormat and a streaming Sink to turn… | ||
| A look at Flink’s internal data structures & algorithms for efficient checkpointing | Stefan Richter | 28m |
Flink implements incremental checkpoints that record only deltas and asynchronous checkpoints that let the pipeline keep running; those features hinge on… | ||
| TouK Nussknacker – creating Flink jobs with GUI | Maciek Próchniak | 35m |
TouK Nussknacker is a GUI platform that builds real Flink jobs from a model jar and JSON process definitions, used in production for ~1 year at a large Polish… | ||
| Looking for patterns with Flink CEP library | Dawid Wysakowicz | 22m |
Flink CEP 1.3 demonstrates concrete pattern features: optionals, quantifiers (one or more / unbounded), and kleene closures applied to clickstream and… | ||
| Time-Based Aggregates: The Good, the Bad and the Ugly | Frank Lauterwald | 38m |
Time-based aggregations on Flink break when Kafka partitions are skewed or replayed and watermarks lag, causing operators to stall or drop late events; details… | ||
| Tink, a temporal graph analytics library for Apache Flink | Wouter Ligtenberg | 16m |
Tink is a Java library built on Apache Flink’s batch API and Gelly that represents edges with start and end timestamps as interval graphs and implements… | ||
| Fast Data at ING | Martijn Visser, Bas Geerdink | 46m |
ING builds a production streaming data platform using Apache Flink and Kafka that delivers actionable notifications and fraud detection. The platform processes… | ||
| Connecting Apache Flink and the Pravega Stream Store | Flavio Paiva Junqueira, Stephan Ewen | 46m |
Pravega is a stream storage that stores unbounded streams by splitting them into mutable segments that can auto-scaling segments through split and merge and… | ||
| Efficiently Executing R Dataframes on Flink | Andreas Kunft | 22m |
A prototype embeds the fastR R implementation on the JVM using Truffle/Graal so R user-defined functions run inside Flink task managers instead of in an… | ||
| Apache Flink meets Apache Mesos and DC/OS | Till Rohrmann | 42m |
Apache Flink runs as a Mesos framework using a Fenzo scheduler for offer-to-task matching, a Flink Mesos dispatcher for high availability, and per-job… | ||
| From Streams to Tables and Back Again: A Demo of Flink’s Table & SQL API | Timo Walther | 23m |
Live demo of Flink’s Table & SQL API shows a prototype SQL CLI that runs in embedded and cluster modes, reads a JSON catalog, and registers user-defined table… | ||
| Talk Python To Me: Stream Processing in your favorite language with Beam on Flink | Aljoscha Krettek | 41m |
Demonstrates a running design that lets Python Beam pipelines execute on Flink by cutting Python PTransforms out of the graph and running them in a separate… | ||
| StreamApprox: Approximate Stream Analytics in Apache Flink | Pramod Bhatotia, Do Le Quoc | 30m |
Prototype implements online stratified reservoir sampling for Apache Flink (also ported to Spark) and answers linear queries like sum and mean over streams… | ||
| Integrating Flink and Kafka in the standard genomics pipeline | Francesco Versaci | 19m |
A Flink pipeline replaces the initial genomics steps with two loosely linked modules that stream through Kafka: reader and writer modules exchange many… | ||
| Closing Session | Kostas Tzoumas | 08m |
Conference closes with concrete takeaways: announces a new product from data Artisans with an early‑access program, cites an open‑source metadata management… | ||
| nothing matches | ||