It’s finally here: Python on Flink & Flink on Zeppelin
summary
PyFlink exposes PyFlink 1.9 Table API and lets users pip install pyflink for Python 3.5–3.7, while native Python UDFs via Apache Beam portability arrive in 1.10. Architecture reuses Java operators and aligns table/tableenv objects to avoid earlier 30x slowdowns; connectors for Kafka and JDBC and support for pandas/Series are available. Zeppelin integration runs interpreters that manage dependencies, visualize results, submit Flink jobs, and resume from savepoints with changed parallelism. Roadmap includes an ML pipeline API and Alink-based machine learning fully runnable from PyFlink.
AI-written summary · about
chapters (10)
- 0:00Introduction to PyFlink
- 2:50PyFlink Design and Goals
- 7:04PyFlink Architecture Details
- 10:18Job Execution and UDFs
- 14:26Using PyFlink
- 17:39Advanced Features and Demo
- 22:12Apache Zeppelin Overview
- 26:22Zeppelin Features
- 28:12Zeppelin Demo
- 34:35Job Control and Conclusion
related talks
35mPyFlink: Building Flink applications with Python language
25mBuilding a Python Data Pipeline with Apache Flink
41mTalk Python To Me: Stream Processing in your favorite language with Beam on Flink
42mWhat's new for Flink's Table & SQL APIs? Planners, Python, DDL
39mInteractive Session: Beam and Beam on Flink general roadmap discussion and AMA
38mBeam on Flink: How does it actually work?
40mFrom concept to production; the fastest way to develop & deploy Apache Flink jobs
43mLeverage the power and simplicity of SQL on Apache Flink