Leif Wickland Interview

on YouTube ↗04m

" summary

Performance is the limiter: Python prototypes are orders of magnitude too slow for real-time ad tech, so the stack relies on Scala and a Spark shell pointed at a big data lake to distribute ad hoc searches and debug production issues. Data volume keeps doubling every few months, forcing constant cost reduction for processing. Strongly typed functional programming reduces regressions and on-call paging, and practical tools mentioned include a Vegas plotting library and an ML library that immediately solve recent analysis gaps.

AI-written summary · about

" chapters (6)

1. 0:00Intro
2. 0:18Engineering issues
3. 1:07Ad hoc analysis
4. 1:48Current issue
5. 2:26Functional programming
6. 3:03Outro

" related talks

• 53mNoLambda: A new architecture combining streaming, ad hoc, machine...E. Chan · Philly ETE 2016
• 07mInterviewPaul Snively · Scale By The Bay 2018
• 05mInterviewNimbus Goehausen · Scale By The Bay 2018
• 05mInterviewAdil Akhter · Scale By The Bay 2018
• 05mInterviewNick Pentreath · Scale By The Bay 2019
• 19mThe Laws of ML at ScaleKetan Umare · AI @Scale 2022
• 04minterviewUmayah Abdennabi · Scale By The Bay 2019
• 47mScala: The Unpredicted Lingua Franca for Data ScienceDean Wampler · GOTO Chicago 2016