Scala: The Unpredicted Lingua Franca for Data Science
summary
Argues that Scala becomes a practical lingua franca because core big‑data tools like Spark use it, then demos working code in Spark notebook demos and the Scala REPL to build RDDs, DataFrames and SQL queries from JSON airport data with caching. Shows word‑count, plots a log‑log power law and finds a concrete surprise: 25 primes out of 100 (75 nonprimes), uses binary‑packed DataFrame rows and generated bytecode in Spark 2 for big speedups, and notes real weaknesses: thinner Python/R libraries and JVM memory/GC limits.
AI-written summary · about
chapters (33)
- 0:00Intro
- 5:23Counting Words
- 7:35Plotting Words
- 11:07Functional Programming
- 17:35Functional Methods
- 18:27Map
- 20:24Spark
- 23:32Scala Interpreter
- 24:14Scala Records
- 24:40Python vs Scala
- 25:25Pattern Matching
- 25:47Pattern Management
- 27:02Prime Numbers
- 27:26Pattern Match Expressions
- 28:18Home Requests
- 29:35Spark Example
- 30:59Other Languages
- 31:20Statically typed
- 31:50Record type
- 32:58Type signature
- 33:25Dynamically type
- 33:42Java vs Scala
- 36:07Scala Python API
- 36:36Using the sequel context
- 37:49Using the sequel API
- 40:17DataFrames API
- 40:28Time
- 40:58Recursion
- 43:00Data Libraries
- 44:16Garbage Collection
- 45:15Dataframe API
- 46:19Spark to
- 46:41Good abstractions
related talks
42mScala: The Unpredicted Lingua Franca for Data Science
47mScala: The Unpredicted Lingua Franca for Data Science
01h 12mBig Data in Java World: Apache Spark and Scala. Arch and live coding
45mScala and the JVM as a Big Data Platform Lessons from Apache Spark
46mScala and the JVM as a Big Data Platform
44mData Science at Scale with Apache Spark
33mThe state of Scala as a platform for statistical computing
01h 05mScala: The Simple Parts