Beyond Significant Terms #bbuzz
summary
An open‑source intelligence pipeline using RabbitMQ, Elasticsearch and a graph DB ingests noisy multilingual public data and applies significant terms aggregation plus a custom TF×IDF‑style score to fix the default method’s failure on single documents (which otherwise returns rare background tokens alphabetically). Entities come from a wide‑recall capitalized‑term regex scored by domain‑IDF scoring and then reconciled against DBpedia/Wikipedia; trending uses smoothed time windows and cross‑source correlation. System runs in real time, is faster than the built‑in significant‑terms query, and succeeds under tight budget and low‑resource language constraints.
AI-written summary · about
chapters (20)
- 0:00Intro
- 0:55Overview
- 1:35Elastic Search
- 3:59Simplicity
- 5:27Magic
- 6:02Significant
- 7:29JLH
- 8:53New Score
- 10:24TFIDF
- 11:45Finding things in text
- 12:06Challenges
- 13:34Summary
- 14:47Reranking
- 17:13Selfdevelopment
- 17:35Trends
- 18:06Smoothing
- 19:11Filter
- 19:26Relevant Documents
- 19:37Significant Terms
- 20:58Outro
related talks
41mGetting Started with Query Understanding #bbuzz
31mTaking AI/ML to the next level: Text!
37mBM25 demystified #bbuzz
47mThe Neural Search Frontier #bbuzz
29mEnabling the world to Search by Meaning
01h 26mBagging to BERT- A tour of applied NLP
43mSentiment Analysis in Action: Building Your Real-time Pipeline
34mExposing Dark Data in the enterprise with custom NLP