The Post-Modern Data Stack for AI: Open Source Foundations for a Multimodal La...

on YouTube ↗16m

" summary

Lance format is a new open source lakehouse storage that stores images, audio, video, embeddings and tables in the same table on object storage, designed for AI workloads and claimed to support tens to hundreds of petabytes. It delivers 100x faster random access than Parquet for vector-style lookups, enables zero-copy data evolution so columns and embeddings can be added or backfilled incrementally, and is used by teams like Netflix for a multimodal media data lake.

AI-written summary · about

" chapters (11)

1. 0:00Introduction
2. 1:04AI product differentiation
3. 2:08AI vs traditional data challenges
4. 3:55Traditional lakes limitations
5. 5:21New lakehouse for AI
6. 6:24Netflix multimodal case
7. 8:11Open source foundation needed
8. 8:32Lance solves AI data issues
9. 12:49Lance in AI infrastructure
10. 13:10AI data trends
11. 15:19Q&A

" related talks

• 28mBeyond Parquet and ORC: Upgrading Data Infrastructure for Multi-modal AI with Lance Col...Chang She · AI.dev: Open Source GenAI & ML Summit North America 2023
• 31mTransforming Multimodal Data Management with LanceDB-RayLei Xu, Change She · Ray Summit 2024
• 31mBuilding a Data Infrastructure for AI/MLKeith Pijanowski · AI_dev: Open Source GenAI & ML Summit Europe 2024
• 31mAgents Need More than Vector SearchPrashanth Rao · AI By the Bay 2025
• 28mLanceDB: lightweight billion-scale vector search for multimodal AIChang She · PyData Global 2023
• 27mEmpowering PySpark with Lance Format for Multi-Modal AI Data PipelinesLu Qiu, Allison Wang · PyData Global 2024
• 11mRecent trends in data and machine learning technologiesBen Lorica · Strata Data Conference 2019 - New York, NY
• 50mOleksandra Bovkun | Data Track | Compass AI & Data Summit 2025Oleksandra Bovkun · Compass AI & Data Summit 2025