~/talk

Holden Karau, IBM | BigDataNYC 2016

Talkat BigDataNYC 2016 · 2016-09-26..09-29· [machine-learning, large-language-models]
slides:none on file

Spark 2.0 migrates core work onto the Dataset API, removes legacy “junky” interfaces and enables better spilling to disk so large partitions run without OOMs; structured streaming exposes SQL-style queries on streams so aggregations and streaming ML run without manual micro‑batch plumbing. Notebooks plus Spark let data scientists operate on full-scale data (so sampling declines) and enable dataset‑recommendation workflows and rapid model retraining and deployment across production pipelines.

AI-written summary · about

  1. 0:00Introduction and Boo Boo
  2. 1:06Spark 2.0 and API cleanup
  3. 3:55Spark's simplicity and usability
  4. 6:19Open source collaboration in tools
  5. 8:44Data scientists vs developers
  6. 10:10Machine learning with Spark
  7. 12:20Sampling is dying
  8. 13:25Big data tool integration
~/search
type to search 6,518 conferences, 143,918 speakers and 192,133 talks
~/help
cmdctrl k
search every conference, speaker and series
/
focus the filter search
h
home
c
conferences
s
speakers
e
series
t
light / dark — the status bar button does the same
← →
on a talk: 5 seconds back / forward
0–9
on a talk: jump to 0%–90% of the recording
?
this help
esc
close this, then clear every filter — or just the search, while you are typing in it

Ignored while you are typing. Pages without a filter rail have no search to focus.