How Companies are Using Spark, and Where the Edge in Big Data Will Be
summary
Apache Spark (Apache/Berkeley) aims to push value from data by improving speed and algorithmic power on commodity Hadoop clusters. Claims include up to 100x faster than MapReduce, SQL and BI via Shark so tools like Tableau work interactively, and machine learning where a model trains on 100 million records in 30 minutes. Real-time use includes Spark Streaming switching video streams for 4 billion streams/month and business workflows that merge sources in seconds to minutes.
AI-written summary · about
chapters (8)
- 0:00Start
- 1:22Implication
- 2:23Two Factors
- 5:401. Yahoo! Personalization Example challenge: relevance of news stories
- 7:10Yahoo! Ad Analytics
- 8:403. Conviva Real-Time Video Optimization
- 9:38Get Started
- 10:08Conclusion
related talks
45mFrom Big Data to Fast Data: Apache Spark
44mData Science at Scale with Apache Spark
59mSpark 2.0
01h 25mApache Spark for Big Data Processing
06mUnleashing intelligence and data analytics at scale
01h 48mBig Data made easy with a Spark (3.0)
10mNew Directions for Spark in 2015
17mMachine Learning at the Speed of Light with Spark