Building data lakes for your customers

on YouTube ↗49m

" summary

Shows four concrete extraction patterns and working demos: database-level CDC via JDBC or third-party log readers (Lambda/Glue connecting to HANA, pulled ~5,602 rows as JSON), application-level IDoc or RFC push through API Gateway + Lambda to S3 (XML IDoc received and stored), ODP extractors with skip and delta tokens (chunked full loads of 4,000 then 1,602 rows, then single-record delta), and SLT push to a staging DB or S3. Includes code samples and blog links.

AI-written summary · about

" chapters (16)

1. 0:00Introduction
2. 1:13What is data
3. 1:41Accessing data
4. 2:07Data generation
5. 2:38Reference architectures
6. 4:12AWS services
7. 6:24Extract Data
8. 7:40Pull Data
9. 10:15Base Level Extraction
10. 11:51Demo
11. 16:06Application level extraction
12. 19:11Simple demo
13. 20:45Integration blog
14. 27:18Operational data pruning
15. 31:00Demonstration
16. 35:55Recap

" related talks

• 41mTransactional data lakes for your analytical workloadsMichael Greenshtein, Eliran Efron, Alon Agmon · AWS Summit Tel Aviv 2023
• 20mDelta Lake: Turn your Data Lake into a LakehouseVinoaj Vijeyakumaar · DataEngBytes 2022
• 25mBuild open format data lakehouseSatish Mane, Rajeev Jahswal · Conf42 Cloud Native 2023
• 45mModern Data ArchitectureAlexey Grishchenko · JavaDay Kyiv 2015
• 45mThe Data Lakehouse: A tech talk for everyone who loves dataFrank Munz · Voxxed Days Ticino 2023
• 30mRaw Data to Business Insights: What You Need to Build a Modern Data LakeJavier Ramirez · AWS Summit Online EMEA 2020
• 59mModern data lakes with Lake Formation, Amazon Redshift & AWS GlueRaghavarao Sodabathaina, Changbin Gong · AWS Summit Atlanta 2022
• 29mData Acquisition: Overcoming ChallengesArtsiom Yudovin · Big Data Conference Europe 2023