Building end to end streaming analytics serverless pipeline

on YouTube ↗41m

" summary

Covers building a serverless streaming data pipeline on AWS that uses AWS streaming services, Apache Flink for stream processing, and Amazon OpenSearch for downstream analytics and dashboards. Explains how to consume data from various client applications and implement scalable, serverless end-to-end analytics workflows without managing infrastructure.

AI-written summary · about

" chapters (12)

1. 0:00Start
2. 5:38The foundation of streaming storage systems
3. 6:20The commit log is the construct for real-time data stream
4. 9:21Amazon Kinesis Data Streams
5. 11:11Amazon Managed Streaming for Apache Kafka
6. 13:38Stream Processing - State
7. 15:49Types of delivery and processing semantics
8. 17:43Time Semantics
9. 18:58Windowing Patterns
10. 21:12Apache Flink
11. 23:29Amazon Kinesis Data Analytics (KDA)
12. 40:43Data & Analytics Track

" related talks

• 39mLarge scale streaming analyticsJay Palaniappan · Data Con LA 2019
• 32mBuilding and scaling a log analytics platform - a serverless approachNarendran · Rootconf Cloud Server Management Miniconf Chennai 2017
• 46mServerless Deployment of AnalyticsJeffrey Jensen · Global IoT DevFest 2017
• 35mStreaming and Real-Time AnalyticsDonnie Prakoso · AWS Summit Online 2020
• 30mBuild and run streaming applications with Apache Flink and Amazon Kinesis Data AnalyticsHausmann · Flink Forward Europe 2019
• 49mBuilding an Enterprise Data Lake Architecture on AWSStephan Schiller, Alena Schmickl · Porto Tech Hub Conference 2024
• 24mStreaming data analytics with Apache FlinkMasudur Rahaman Sayem · DataEngBytes 2021
• 35mStreaming and Real-time AnalyticsDonnie Prakoso · AWS Summit Online EMEA 2020