Professional Data: Using Elasticsearch's Aggregation Framework
summary
Shows concrete Elasticsearch work on real datasets: geospatial UK housing summarized with geohash precision five towers sized by median house price (median avoids average skew), joined to crime data to find theft rises in affluent cells, Enron email graphs built with a breadth‑first collect model, MovieLens recommendations using significant terms where Talladega likers jump a tag from 0.5% to ~20%, Amazon reviews reindexed into an entity centric index via Groovy updates to spot coordinated five‑star “fanboy” accounts, and MOT records used to detect odometer and taxi testing anomalies; most queries run subsecond (example 30 ms).
AI-written summary · about
chapters (23)
- 0:00Introduction
- 1:45Search interface 20
- 2:57Analytics interface of today
- 4:27Optimised for real time
- 6:15Worked examples
- 9:41UK Housing data: percentiles
- 10:51UK Housing data: terms
- 13:56Geo as a common link between datasets: housing crime
- 15:13Connected data: Enron emails
- 22:34Recommendations: MovieLens data
- 23:05Random samples should hold no surprises
- 24:38Non random sample: people who liked Talladega nights
- 26:11Problem: avoid analysis of poorly focused sets
- 27:31How do we get a smaller, representative sample of users?
- 28:55Putting search and analytics together..
- 32:37Amazon marketplace reviews
- 35:58Anatomy of an entity indexing groovy script
- 38:33Drilling down into seller #187's fanboys
- 39:15UK car roadworthiness test: raw data
- 39:37Derived car attributes
- 41:29Miles driven vs number of days for fix
- 43:19In summary
- 44:24Questions?
related talks
41mBetter Data Exploration and Visualisation with Elasticsearch and Kibana
42mMake Sense of your (BIG) data!
01h 01mData magic with the Elastic stack!
39mHandling, extending and presenting data with ELK+
33mRelevant Data Analysis: Apache Solr Analytics #bbuzz
55mSocial Listening for Richer Data, Unstructured Analytics
45mMake Your Data FABulous
38mTransforming Data to Unlock Its Latent Value