Data as culture as software
summary
Six years at Pinterest show how logging and engineering shape usable metrics: raw request logs land in Kafka/Redshift, small derived tables (xd7/xd28) replace heavy queries, and a rule-based table called possible_spammers_data plus ML filters removes millions of fake accounts that skew signups. An iOS 3.0 background fetch initially explains a DAU jump — engineers estimate a 3% effect from feeds home but deeper analysis and an explicit app state ping reveal ~99% of the spike was background activity, so counting and opinionated datasets win.
AI-written summary · about
chapters (14)
- 0:00Start
- 3:01Pinterest's journey
- 4:12varnish
- 7:02Daily Active Users
- 10:18Derived tables: DAU
- 11:33How is next week different?
- 20:16Spam filtering
- 24:58Daily Active iPhone Users
- 36:07Data chaos: JSON
- 38:26data governance
- 39:19Spectrum of certainty
- 39:59A/B testing
- 40:30Software tools
- 41:11Data is unruly
related talks
35mBuilding Data-Intensive Teams
47mKeynote - The Evolving Role of Data in Software Development
13mData for product
28mGenerate Insights, Not Excuses: How your data is slowing you down
33mAccidental DataOps: How we matured our organization with data, and how you can too!
21mBig Data and You: Four Ways to Use Data Every Day -
47mData Analytics: Realizing the Full Potential
22mData Analysis in the Age of Data Democratization