~/talk

Democratizing the pipeline with Apache Beam and Dataflow

on YouTube 10m
Talkat Strata Data Conference 2017 - New York, New York · 2017-09-25..09-28· [data-engineering, machine-learning]

speakers

name:Tyler Akidau
links:none on file
name:Reuven Lax
links:none on file

resources

video:https://youtu.be/izhz1O0DMMo
slides:none on file

summary

Describes Apache Beam as a portability framework and programming model for Java and Python pipelines and Google Cloud Dataflow as a managed runner that auto-scales and hides worker details. Shows runnable backends: Cloud Dataflow, Apache Flink, Spark, Apex; names Spotify as an early user. Promises soon: run Python pipelines on any runner via cross-language SDKs, mix-and-match SDKs and runners, better autoscaling and auto-tuning, and streaming SQL across long-lived pipelines.

AI-written summary · about

chapters (5)

  1. 0:00Introduction
  2. 0:30What are Beam and Dataflow
  3. 2:38Industry leaders
  4. 3:43Future of Beam
  5. 6:55Future of Google

related talks

~/search
type to search 6,249 conferences, 138,987 speakers and 185,172 talks
~/help
cmdctrl k
search every conference, speaker and series
/
focus the filter search
h
home
c
conferences
s
speakers
e
series
t
light / dark — the status bar button does the same
?
this help
esc
close this, then clear every filter — or just the search, while you are typing in it

Ignored while you are typing. Pages without a filter rail have no search to focus.