Discovering Research Ideas Using Semantic Vectors & Machine Learning
summary
Unsilo ingests publishers' full text with an Apache UIMA pipeline plus Python/Java tooling, runs heavy NLP and graph analytics, and trains word2vec‑style embeddings augmented by ontologies to collapse “100+ million” raw phrase occurrences down to ~2–3 million normalized concepts. The system extracts longer phrase tokens, computes local/global features (TF deviations, concentration, document centrality), builds human readable fingerprints, serves related‑content widgets and timelines using forward/back citation pruning, and claims improved, unbiased discovery at publisher scale (clients with 10–15M articles).
AI-written summary · about
chapters (16)
- 0:00Introduction
- 0:36UNSILO Mission
- 2:47UNSILO Core Technology
- 3:25Key Challenges
- 5:14Full Text Search
- 5:49Using Keywords and Ontologies
- 6:04UNSILO Exhaustive Concept Extraction
- 6:40UNSILO Complete Semantic Mapping
- 13:04Phrase Extraction
- 19:11Word Embeddings and Word2Vec
- 20:49Ontology Augmented Vector-space
- 21:11Synsets built from Vector Cosine Similarity
- 23:40Human readable Fingerprints
- 25:23UNSILO Discovery Widgets
- 29:23Easier Content Exploration
- 31:18Future Directions
related talks
31mTaking AI/ML to the next level: Text!
42mThe Machine Understands Text — Learning and Search at Micro and Macro Scale
46mNatural Language & Search, Improve your Search w/ Vector capabilities
32mBuilding semantic search using Deep Learning
22mThe Capabilities of Semantic Search
42mResearch in the Age of AI
30mSemantic Search and Neural Information Retrieval
13mA Beginner’s Guide to Semantic Search