Big Data, Disruption And The 800 Pound Gorilla In The Corner
summary
Claims that volume is largely solved by commercial data warehouses running parallel column stores, but cloud migration (S3, Snowflake) is the big disruptor. Velocity needs newSQL in-memory engines (VoltDB, MemSQL) or CEP/Kafka for millisecond trading, not MapReduce/Hadoop. Data science hinges on fast large-scale matrix multiply and ML libraries (scikit‑learn, TensorFlow) yet deep learning requires massive training sets and is often unexplainable. The real 57-stone gorilla: data integration consumes ~80–99% of data scientists’ time; enterprises (GE: 75 procurement systems, $100M saving) need ML at scale to automate cleaning and matching.
AI-written summary · about
chapters (10)
- 0:00Introduction and overview
- 1:46Big data volume and warehouses
- 4:19Cloud disruption in data warehousing
- 7:31Business intelligence to data science
- 14:50Hadoop’s decline and MapReduce
- 18:07Big data velocity challenges
- 22:05High-speed transaction systems
- 25:40The data variety gorilla
- 35:25Future of data integration
- 36:09Summary and closing
related talks
58mUnderstanding Big Data for Software Engineers
02mThe Future of Data Science/ML at Scale
41mData Science – The New Troublemaker in the Delivery Pipeline
47mWhat's Ahead for Big Data?
11mRecent trends in data and machine learning technologies
08mRevolutionizing Healthcare with Big Data and ML
58mTales of Data Architecture Evolution
46mBig Data. Or Just Too Much Data?