Software Quality As It Relates To Data
summary
Argues that modern data platforms on AWS work if systems are loosely coupled and governed, not monoliths, and gives concrete tooling: S3 as the scalable data lake (claims 11 nines durability, four nines availability), Redshift, Athena, EMR, OpenSearch, Kinesis/MSK, Lake Formation, IAM, GuardDuty and Inspector. Recommends decoupling storage from compute, using serverless for infrequent jobs, applying master data management rules automated with ML, and warns a data lake becomes a swamp without purpose and cataloging.
AI-written summary · about
chapters (9)
- 0:00Introduction
- 3:51Data lifecycle in software
- 8:43Modern data architecture
- 15:32Compliance in government data
- 19:30Scalable data lakes
- 27:31Data access security
- 33:28People process technology
- 37:54Master data management
- 41:09Decoupled architectures
related talks
53mHow to build a modern data architecture
37mBuild a Modern Data Platform on AWS
42mModern Data Strategy on AWS
18mFive data lake considerations
25mPlatform Engineering for Modern Data Infrastructure
47mBuilding Data Lake Platform Fully Serverless
24mModernizing the Data Warehouse
43mAnalytics capabilities architectural patterns: A journey for data-driven insights
