Analytics at scale: Migrating Intuit to AWS with cloud-native tools
summary
Intuit moves hundreds of sources from on‑prem to S3, lifts its MPP to EC2, then modernizes by shifting ETL left onto Amazon EMR and AWS Glue, and exposes the lake for interactive SQL via Amazon Athena after enabling Hive metastore integration. A custom Athena Federated Query connector lets Athena join the lake with the MPP; analysts develop in Athena and run production via Presto. Amazon Redshift hosts facts/dimensions, uses Redshift Spectrum to query S3 and Redshift Data Sharing across ~8–9 clusters.
AI-written summary · about
chapters (18)
- 0:00Introduction
- 1:52What is Intuit
- 2:43Data story
- 5:30Lift and shift
- 6:53Hive Metastore
- 7:37Objectives
- 8:45Data flow
- 9:15Getting there first
- 11:09federated query
- 13:02AWS blog post
- 13:44Leveraging the data lake
- 16:22Partner with your users
- 18:13Adoption of AWS AMER
- 20:07AWS Glue
- 22:14Thinking differently
- 24:12Analyst owned pipelines
- 27:18Redshift
- 30:51Recap
related talks
28mTurn Data Into Insights
29mGrowing insights through hybrid data architectures
30mRaw Data to Business Insights: What You Need to Build a Modern Data Lake
27mModernize Your Data Warehouse
29mHow financial services companies derive greater value from data with AWS
22mHow Rocket Companies modernized their data strategy on AWS
01h 02mData warehousing reinvented for today’s needs
21mOptimizing data delivery for faster, better analytical insights