The challenges of data science for veteran media organizations
summary
Small team modernizes a legacy newspaper into a tech product by rebuilding pipelines, warehouses and tooling rather than chasing hype. Core stack centers on S3 + Airflow + Databricks with Redshift Spectrum, BigQuery and Tableau for analytics; Docker standardizes deployments. Day-to-day is heavy ETL — about 50–60% ETL time — and data journalists spend 30–40% building infra. Built an automated quake-publish bot and a dynamic paywall model to score likely subscribers and drive conversion.
AI-written summary · about
chapters (16)
- 0:00Start
- 3:21Introduction
- 6:20What's important and what's not
- 9:01Why care about Data / Data Science?
- 9:27Measure all the things
- 11:43Make the bosses care
- 12:30Why it's easier than ever
- 13:10Structure of a data stack
- 13:27Data terms
- 16:16The reality
- 18:38Do what works for your team
- 21:11Tools - Storage
- 22:15Tools - Presentation
- 23:34Tools - Software
- 24:29Tools . Data Engineering
- 25:25Tools - Data Science
related talks
30mData Engineering building your BI infrastructure from scratch
42mWeb Development To Big Data. A Journey
34mTales of Data Architecture Evolution
35mBuilding a data analytics platform on strong opinions held loosely
39mBuilding a Modern Data Warehouse Without Breaking the Bank
29mHow I built an entire data platform by myself that thousands of data engineers use in one year
17mDemocratising data analytics
01h 17mData Engineering in the AI Era: Building a Robust Foundation for AI Applications