Scala: The Unpredicted Lingua Franca for Data Science
summary
Claims Scala becomes the practical engine behind distributed data science by enabling interactive, production-ready workflows using Spark Notebook and Docker. Demonstrations run a prime-counting parity that shows a ~3:1 non-prime:prime ratio, load a 15–20 year public flights JSON to show schema inference and SQL-like grouping via DataFrames, and move the same collection code into parallel RDD execution. Examples highlight tuples, pattern matching, case classes, and seamless transition from exploration to deployable Spark jobs.
AI-written summary · about
chapters (19)
- 0:00Intro
- 0:36Why Data Science
- 1:38Big Data
- 2:14Business
- 4:49Hadoop
- 6:28Spark
- 7:54Bit Data Computing
- 9:41Why Scala
- 9:54Mathematica Notebook
- 11:25Python vs Scala
- 12:51Examples
- 17:26Scala Case Classes
- 19:43Scala Data Frames API
- 21:26Scala Sequel Query
- 23:52Distributed Data Science
- 29:54Team
- 35:47What is missing
- 37:28What we can do
- 38:22Other initiatives
related talks
47mScala: The Unpredicted Lingua Franca for Data Science
32mWhy Use Scala in a Bigdata Context?
44mData Science at Scale with Apache Spark
36mData pipelines engineering made simple with Scala
33mThe state of Scala as a platform for statistical computing
24mBig Data Science in Scala
42mScalable Data Science in Python and R on Apache Spark
01h 48mBig Data made easy with a Spark (3.0)