Fast and scalable domain-specific knowledge graphs generation
summary
Builds three domain KGs: a social KG that ingests the Twitter firehose, extracts users links topics and time, uses pseudo‑relevance feedback and a hashtag index to auto‑generate Wikipedia‑style stories that the author claims match Wikipedia quality and powered results in Bing; an e‑commerce KG that fuses signals with a voting across sources scheme, query logs, k‑means clustering and deep semantic models to derive brands categories and products; and a healthcare KG that ingests SNOMED/RxNorm/MeSH, targets human-in-the-loop QA, and runs on Cosmos and Scope for scale.
AI-written summary · about
chapters (14)
- 0:00Start
- 0:46Introduction Knowledge graph (KG) describes objects of interest and connections Organizing data as nodes and edges Examples Microsoft Satori Microsoft, Google Knowledge Graph, Amazon Product Graph Knowledge bases (Bs): Yago, Freebase Knowledge graph and knowledge base terms are used interchangeably Our focus today is more on the construction than reasoning
- 2:09Introduction - || Interested in scalability and fast generation How to identify nodes and derives edges So far, most research on KGs/KBS use Wikipedia as source Benefits easy to read, easy to parse. Wikipedians Drawbacks coverage, outdated content, bias What do we do when there is no Wikipedia? and there are plenty of examples in the real world
- 3:28Some observations on Twitter Drinking from a firehose is hard Low value at the atomic level Potential to build new artifacts Aggregations Knowledge graphs Applicable to other social networks (e.g., Linkedin, Facebook, etc.)
- 4:29Social Knowledge Graph Input: Twitter firehose Output: a knowledge graph Components Links Topics Entities leg, people, organizations, places Time Look at the graph from those views Focus on high-quality content Relevant content from trusted users on good topics
- 9:45Vote Data Structure Problem with basic counts Artificial popularity Aggregate counts for tweets Compute counts that capture Elements in-gram, hashtag link Connections hashtag-link, n-gram-link Frequencies
- 16:48Challenges Very dynamic domain (brands and products appear/disappear) Lack of major sources with clean brand/product data Hard to define and detect products Distinction between brand and product is sometimes blurred Retailers don't always provide clean data
- 17:36Brands and brand domains Voting mechanism Use as input a number of sources Each source assigns 1 vote to the presence of a term in their respective sources Compute voting above a threshold as final score Use query logs to extract most frequent URL @p1
- 18:38Brands and brand domains Voting mechanism Each source assigns 1 vote to the presence of a term in their respective sources Compute voting above a threshold as final score Use query logs to extract most frequent URL @p1
- 20:35System design New data sources can be added without affecting the existing process Bottom-up strategy with a focus on simplicity and data cleaning Implementation in Scope/Cosmos
- 21:19Healthcare domain Scientific medical knowledge Existing taxonomies and data sources Examples SNOMED (Systematized Nomenclature of Medicine) RxNorm (medications available on the US market) MeSH (Medical Subject Headings)
- 22:03Challenges Very sensitive data EMR (Electronic Medical Records) Clinical relevance Vocabulary mismatch MDs describing a diagnosis Data labeling and curation
- 24:21Conclusion Active area of in academia and industry CACM article August 2019 Many workshops at top tier conferences Combination of many techniques Domain expertise
- 26:20Approach Unsupervised Focused on data quality and simplicity Generate brands using data fusion and voting Tag brands with categories Generate products using different techniques
related talks
41mData Science Education. Building Knowledge Graphs
28mCreating Enterprise Knowledge Graphs from Unstructured Data
01h 40mKnowledge Graphs & GraphRAG: Techniques for Building Effective GenAI Applications
30mHow Graph Data Science Can Turbocharge Your Knowledge Graph
30mMaking Your Knowledge Graph LLM-Ready: Quality Assessment and Design Strategies for GraphRAG
36mGeorgia on my Mind: NLP Meets Social Network Analysis for Exploring New Domains
18mFree Texts to Knowledge Graphs - Holy Grail of Semantic Computing on an Industrial Scale
05mAugmenting Visual Information in Knowledge Graphs for Recommendations