What's new for Flink's Table & SQL APIs? Planners, Python, DDL
summary
Flink 1.9 ships a pluggable Blink planner and a split table-module architecture so users can opt into Blink while falling back to the old planner; the unified TableEnvironment replaces many prior environments. A new logical type system defines ~24 types and stronger type inference; SQL DDL and Hive catalog integration are added. A Python Table API is available (manual build today, UDFs via Beam portability planned), and efforts target end-to-end SQL performance including TPCH/TPC-DS compatibility.
AI-written summary · about
chapters (18)
- 0:00Intro
- 2:38Apache Flink's Relational APIs
- 6:20Architecture before Flink 1.9
- 7:46Architecture in Flink 1.9+
- 8:23Alibaba's Contribution of Blink
- 9:51How can we merge Blink gradually?
- 14:15Separation of Concerns
- 15:47Pure Table Programs
- 19:06What is the input and Output Type?
- 20:20New Data Type Abstraction
- 22:07New Type Inference
- 23:49What is Missing in Flink SQL?
- 25:28Introducing the new Python Table API
- 26:45Python Table API Example
- 28:30Defining Sources and Sinks (using builders)
- 30:00Defining Sources and Sinks (using DDL)
- 30:52Running SQL Queries
- 31:23A Preview of FLIP-58: User-defined Python functions
related talks
40mIt’s finally here: Python on Flink & Flink on Zeppelin
43mLeverage the power and simplicity of SQL on Apache Flink
47mTaking a look under the hood of Apache Flink’s relational APIs
23mFrom Streams to Tables and Back Again: A Demo of Flink’s Table & SQL API
39mBetter, Faster, Stronger Streaming: Your First Dive into Flink SQL
35mProduction-Ready Flink and Hive Integration - what story you can tell now?
37mKeynote | Flink Ahead: What Comes After Unified Batch & Streaming?
37mThe Future of Apache Flink