John Akred | Hadoop Summit 2014
summary
Covers why Spark 1.0 and MLlib excite production analytics, how YARN as an enabling layer and Impala/interactive engines broaden Hadoop use, and why metadata/discovery and Trifacta for data wrangling matter for a growing data lake. Gives the concrete sensor example (auto-detecting a new sensor, training a model, deploying alerts) and admits that fully automated pipelines don’t exist yet — predicts wide automation in ~15 years — and flags CloudEra’s sales-heavy strategy and Intel partnership as strategic signals.
AI-written summary · about
chapters (11)
- 0:00Welcome to Hadoop Summit
- 1:10Data science trends update
- 3:12Platform innovations and YARN
- 5:18Big whales and startups
- 7:45Developer community focus
- 9:53Exciting technologies today
- 13:05Data quality and schema on read
- 15:10SQL as gateway to Hadoop
- 17:19Automated data integration
- 18:42Open correlation hunting
- 20:08Future of automated insights
