Raw Data to Business Insights: What You Need to Build a Modern Data Lake
summary
Practical playbook: catalog raw files on Amazon S3, run crawlers with AWS Lake Formation to auto-discover CSV/Parquet schemas and query them serverless with Athena instead of ingesting, and build dashboards with QuickSight. Real customers: Zalando moves 15 PB, 400 teams, 8,000 buckets with a nine‑person team and saves 37% in storage; CAF ingests ~1.7 billion points per train per day; Redshift Spectrum queries exabyte-scale data in ~155 seconds.
AI-written summary · about
chapters (15)
- 0:00Intro
- 1:30A brief opinionated history of data analytics Problem
- 5:05Traditional data warehousing approaches don't scale
- 5:55Modem data analytics 101 - Data Lake Basics
- 7:00A possible open source solution
- 8:38Or a cloud native solution on AWS
- 10:48Why choose AWS for data lakes and analytics?
- 11:23Easiest to build data lakes and analytics A single storage layer (Amazon 53) for all analytics and ML A service to build secure data lakes in days • Deep Integration across analytics and infrastructure
- 12:56in action at zalando
- 15:30Most secure infrastructure for analytics Services for security and gomance Customers need to have multiple levels of security, identity and access management, encryption, and compliance to secure their data lakes
- 16:28Most secure infrastructure: Certifications
- 24:35Most scalable, cost-effective, high-performance infrastructure for analytics
- 25:21Most scalable, cost-effective infrastructure for analytics
- 27:43Let's play a game
- 29:28APN Data & Analytics and Machine Learning Competency Partners
related talks
34mBest Practices for Implementing a Data Lake in Amazon S3
28mTurn Data Into Insights
43mManaging a Data Lake on AWS
22mTurn Data Into Insights
43mData warehousing reinvented for today's needs
19mBuilding modern data lakes on Amazon S3
16mBreakdown data silos - Build a data lake on Amazon S3
21mOptimize and manage data on Amazon S3