Airbus makes more of the sky with Flink
summary
Airbus AirSense ingests ADS‑B position streams (about 12,000 aircraft at 2 Hz -> 24,000 msgs/sec, ~2 billion messages/day) and fuses satellite and ground feeds into a normalized Avro pipeline. The stack uses Flink on AKS with Event Hubs (Kafka API), blob storage/Postgres and Databricks; stateful Flink does event tagging, trajectory smoothing, ETA and real‑time alerts with latencies in the hundreds of milliseconds. Chosen over Spark micro‑batch and Azure Functions, the system scales, checkpoints state to blob storage, and exposes dedupe and provider‑fusion tradeoffs.
AI-written summary · about
chapters (19)
- 0:00Intro
- 0:33Main takeaways
- 2:04Context
- 3:44Services
- 5:17Data
- 6:59Partners
- 7:55Investment
- 8:44Pipeline
- 9:24Summary
- 9:56Data Engineering
- 11:23Project Requirements
- 14:42Implementation
- 16:12Storage
- 16:41POC
- 18:07Data Processing
- 20:09Data Provider
- 21:25Learnings
- 28:38Conclusion
- 29:24Questions
related talks
34mBeaming Flink to the Cloud @ Netflix
07mDeliver In-flight stream processing with Apache Flink on Azure
40mLarge-scale near-real-time (NRT) data analytics platform empowered by Apache Flink
46mSmart Transit: Real-Time Transit Information with FLaNK
47mBuilding Modern Data Streaming Apps with Open Source
40mFlink in Zalando's World of Microservices
41mBeyond the Hype: When Apache Flink Solves Real Problems
43mA Data Streaming Architecture with Apache Flink #bbuzz