Lord of the Spark
summary
Argues that Spark is a memory‑first processing engine that works fine without Hadoop and is practical from Java with Spring, using Java + Spring on Spark, Spark 1.6 with YARN/Mesos, RDDs and DataFrames, S3/HDFS input and SparkSubmit. Demonstrates driver vs executor execution, broadcast vs serialization to avoid shipping large configs, and persist strategies to prevent repeated recomputation. Builds a web service that counts top words per artist and compares lists — it runs and returns concrete results (Britney vs Katy Perry: 6/6 overlap; Pink Floyd vs Britney: 0 overlap), proving the approach.
AI-written summary · about
chapters (7)
- 0:00Introduction
- 10:38Data locality basics
- 21:27Spark language choice
- 31:48Popular words service
- 39:15Broadcast and serialization
- 45:54DataFrame API optimizations
- 51:30Text comparison results
related talks
52mSolving Analytical Problems using Apache Spark
55mSpark Architecture
01h 00mGetting Started with Spark
50mIntroduction to Real-Time Big Data with Apache Spark
46mSpark: One Tool for Everything?
54mHarnessing the Power of Spark & Cassandra within your Spring App
01h 12mBig Data in Java World: Apache Spark and Scala. Arch and live coding
34mHive on Spark #bbuzz