Conceptual Framework for entity integration from multiple data sources
summary
Proposes an iterative ordering algorithm that scores datasets by completeness/overlap/inverse/accuracy to decide which sources to merge first, implemented as a Python implementation, not just pairwise matching. Demonstrates it on a real CRM consolidation: about 64 offices and 15–20 disparate CRMs with millions versus thousands of records, multiple languages and alphabets, and shows the approach reduces wasted comparisons and supports incremental merging. Connects to University of Wisconsin open‑source work (Python/Jupyter) and reports experiments and pending publication.
AI-written summary · about
chapters (9)
- 0:00Start
- 0:48Introduction
- 3:26What Is Entity Resolution and Integration
- 15:08Web Incremental Matching
- 20:35Recommendations
- 21:28Frameworks for Entity Matching
- 23:14Real-Life Example
- 24:50Order of Resolution
- 30:18Conclusions
related talks
41mGetting Your Data AI-Ready
23mThe (Swiss cheese) data conundrum...
59mThe Great Data Debate
35mMix, Match, Merge
49mHow to Combine Data from Different DBMSs and Do It Efficiently
33mLessons Learned on Building Data Aggregators: an engineering perspective
26mMeeting The Evolving Data Intelligence Expectations
11mStrengthening the data value chain