Ted Dunning, Chief Application Architect, MapR

on YouTube ↗27m

" summary

Explains a practical recipe: compute co‑occurrence with the G test (log‑likelihood ratio), rank surprising pairs, and use those scores for recommendations, fraud and anomaly detection, language and genome ID, and document classification. Shows cheap shifts data problems from hard to easy, reports saved tens of millions by a small team using this approach, notes naïve chi‑squared overestimates by 200 orders of magnitude, and argues co‑occurrence often matches or approaches complex models.

AI-written summary · about

" chapters (12)

1. 0:00Start
2. 3:30Most data isn't worth much in isolation
3. 5:30With great scale comes great opportunity
4. 6:18A simple example-security monitoring
5. 7:53Another example-fraud detection
6. 20:06Real-life example
7. 21:38Document classification
8. 22:02Language identification
9. 22:12Species identification
10. 22:27Common Point of Compromise
11. 22:41Simulation Setup
12. 23:44Parametric Simulation

" related talks

• 24mTed Dunning, Chief Application Architect, MapRTed Dunning · MLconf SF 2014
• 33mPeople who liked this talk also liked ... Building Recommendation...Ryan Weald · LA RubyConf 2013
• 52mDevil in the HaystackPing Yan · AppSec California 2015
• 46mHow Cheap Learning (not deep learning) Finds Faults in Distributed SystemsTed Dunning · Jfokus 2019
• 35mTowards Interpretable Data ScienceDaniel Deutch · Logic and Algebra for Query Evaluation 2023
• 26mDiscovering Unwarranted Associations in Data-Driven...Florian Tramèr · MLconf 2016 Seattle
• 40mTools and Tricks from a Pragmatic Data ScientistLucas Javier Bernardi · PyData Amsterdam 2016
• 01h 01mD1T2 - Applying Machine Learning to User Behavior Anomaly AnalysisEugene Neyolov · Hack In The Box Amsterdam 2018