Big Data, Bad Analogies
summary
Claim: marketing analogies like data lake = junkyard and data exhaust is useless fumes mislead architecture choices. All data has schema, and choosing schemaless ingest (Hadoop 64MB chunk scans) trades fast capture for expensive reads and downstream reprocessing. Examples: Pinterest reworked logging and schemas, Google F1 reintroduced global ACID for correctness, eBay runs a 6PB warehouse and a 40PB Teradata appliance where a trillion‑row query returns in ~30s. Amdahl shows added nodes do not fix serialized code.
AI-written summary · about
chapters (30)
- 0:00Introduction
- 0:35Bad Analogies
- 2:55Data Exhaust
- 6:06The Elizabethan Era
- 9:25The Victorian Era
- 10:26The Modern Era
- 11:21Charles Cutter
- 12:05Dewey Decimal System
- 13:19Who Won
- 15:13Organizing Principles
- 17:05Database Analogies
- 17:36Book Analogies
- 18:27Data Life Cycle
- 21:03Technical Debt
- 25:05Database history
- 26:47Join in memory
- 27:18Loose coupling
- 28:46Schema flexibility
- 30:19Data consistency
- 31:31Flexibility
- 32:19No sequel databases
- 34:41Tradeoffs
- 36:01Consistency
- 38:58Relational Models
- 39:36Declarative Language
- 40:57CostBased Optimizer
- 42:48Pig Scripts
- 48:09Singularity
- 49:04Resource Efficiency
- 51:23Workloads
related talks
34mAvoiding Big Data Antipatterns
02h 05mBig Data and the Data Lake from the Analyst Perspective
50mBehind the Hype - Architecture Trends in Data
31mData Warehouses Meet Data Lakes
55mThe Great Data Debate 2022
28mYour data lakes need architecture too!
56mResistance is Futile: What You Need to Know about Big Data
26mModern Analytics: One Size Does Not Fit All