Do Multilingual Embeddings Really Share a Semantic Space?
summary
Finds that pre-trained multilingual embeddings mostly align but unevenly: a tiny retrieval demo sometimes works (dog aligns) and sometimes fails (fire emoji returns Sanskrit neighbors); diagnostics use UMAP projections and nearest-neighbour analyses, tokenisation counts, and translation similarity distributions. Tests across 75 concepts reveal medians near 0.8 yet wide tails; the weakest aligned concept is “flat tire.” Tokenisation fragmentation, Wikipedia training-data imbalance, and semantic ambiguity explain why some languages and concepts drift and break retrieval rankings.
AI-written summary · about
chapters (20)
- 0:00Multilingual embeddings intro
- 1:49Words as vectors
- 3:35Shared semantic space visualization
- 5:23Dog alignment example
- 6:04Fire emoji failure case
- 7:47Retrieval system demo
- 9:10Concrete vs abstract concepts
- 10:52Alignment across languages
- 11:36Translation similarities analysis
- 13:23True alignment meaning
- 14:28Strongly aligned concepts
- 15:32Hardest concepts to align
- 16:13Flat tire case study
- 17:16Tokenization impact
- 18:19Tokenization differences
- 19:02Training data imbalance
- 20:05Meaning ambiguity issues
- 21:09Debugging alignment
- 21:54Benchmarking limits
- 22:38Conclusion
related talks
22mMultilingual embeddings to scale NLP models to multiple...
39mAI-powered Semantic Search; A story of broken promises?
25mSemantic Search
41mSemantic Multimodal Multilingual Similarity Engine
24mRediscover Your Data with a New Multimodal Search Capability
28m"Embeddings: How Computers Learned to Read"
45mHybrid search: Greater than the sum of its parts?
29mHallucination by Design: How Embedding Models Misunderstand Language