Discovery & Analytics with Hadoop, Mahout & Solr

on YouTube ↗46m

" summary

Describes a concrete SDA stack: search served by SolrCloud for sub-second queries and near‑real‑time soft commits, bulk processing and storage on Hadoop, user and analytics state in HBase, and machine learning with Mahout (collaborative filtering, clustering, SVD). Notes practical issues: millions of small files, iterative k‑means costs, and missing experiment management. Gives a measured result: daily A/B experiments raised e‑commerce sales ~5–6% in three months. Offers patterns for routing realtime vs batch work and capturing all queries/clicks for modeling.

AI-written summary · about

" chapters (4)

1. 0:00Start
2. 5:26Search Discovery and Analytics
3. 19:29Where Should the Analysis Be Done
4. 22:58Soft Commits

" related talks

• 40mSmarter Search with Spark, Solr and Machine LearningGrant Ingersoll · Data By the Bay 2016
• 22mSmarter Search With Spark-SolrJake Mannix · MLconf 2016 Seattle
• 53mBig Data: from mammoth to elephantRoman Nikitchenko · JavaDay Lviv 2015
• 40mLevel Up: Everything you need to know about search, ...Grant Ingersoll · Berlin Buzzwords 2016
• 29mThe Data JourneyDoug Cutting · Big Data LDN 2018
• 01h 35mDeep Dive into the Big Data LandscapeDean Wampler, Eva Andreasson · GOTO Copenhagen 2014
• 47mSolving Real & Big (Data) Problems Using HadoopEva Andreasson · GOTO Chicago 2014
• 56mBM25 is So Yesterday: Modern Techniques for Better Search Relevance in SolrGrant Ingersoll · ApacheCon North America 2017