"What's in the news?" #bbuzz
summary
Bluekiwi.de implements a news pipeline that fetches thousands of articles daily, indexes them in Elasticsearch and uses significant terms aggregation together with POS tagging (Stanford), chunking and rule‑based fact extraction to produce normalized fact summaries. The system converts headlines into concise searchable keys, deduplicates similar stories, ranks the top items (aiming for a compact top‑10) and runs live on Unister’s site; results are usable but require iterative rule tuning to catch novel entities and edge cases.
AI-written summary · about
chapters (8)
- 0:00Introduction and overview
- 1:31Why another metasearch engine
- 3:58Introducing news search
- 6:54Finding important news
- 9:21Significant term aggregation explained
- 13:32Natural language processing approach
- 18:06Generating summaries and results
- 20:14Live demo and conclusion
related talks
35mUnderstanding the News around the World with Web Scraping and NLP at Scale
26mUnsupervised learning for news summarisation
38mAutomatically categorizing news articles on nrk.no
19mBig data in the service of reliable news
41mGetting Started with Query Understanding #bbuzz
39mEmphasising Relationships in the BBC's Data Using Technologies of the Semantic Web
20mGain speed and space with NLP in Solr #bbuzz
04mBecoming smarter about credible news