How We Built a 30+ PB Storage for AI and Plan to Keep Growing
summary
Describes building a data storage system holding over 30 petabytes for AI workloads, covering requirements, architecture choices, and operational practices. Discusses technologies used for large-scale storage, data ingestion and indexing, and plans for future scaling and development.
AI-written summary · about
chapters (5)
- 0:00Introduction
- 10:22Storage development iterations
- 17:45Technologies and architecture
- 23:45Results and future plans
- 42:29Q&A
related talks
33mAnalytics Platform at Petabyte Scale
24mA Gentle Introduction to Building Data-Intensive Applications
49mBuilding a 100‑Petabyte Analytics Warehouse
43mKey lessons from building Data and AI systems over the last 26 years…
24mManipulating Data in an Artificial Intelligence World
28mAccelerating AI at Scale with Data
24mObject Storage Driven Machine Language Workloads
08mDesigning Scalable, Secure Data Architectures for AI at Enterprise Scale