Unlock the potential of your data: Optimize your data store architecture...
summary
Claims that GPU training needs sustained 100 gigabytes a second class throughput and per-GPU streaming of 1–10 GB/s, that datasets grow from petabytes to exabytes, and that mismatched storage wastes expensive GPU time. Recommends fast parallel filesystems (RDMA or proprietary clients, NFS is often too slow), a separate high-performance tier plus a low-cost object tier (example: Scality ring on HPE Apollo hardware), and transparent hybrid/edge data management to avoid constant data copies.
AI-written summary · about
chapters (7)
- 0:00Global Big Data Conference
- 2:54CHALLENGES CUSTOMERS FACE
- 6:12THE ADVENT OF BIG DATA AND AI HAS DRAMATICALLY CHANGED
- 12:45DIFFERENT ANALYTICS STAGES REQUIRES DIFFERENT DATA ST CAPABILITIES
- 15:08THE DATA ARCHITECTURE TO SUPPORT ENTERPRISE AI AND ANAL NEEDS MUST BALANCE PERFORMANCE AND CAPACITY
- 23:52WHAT'S REQUIRED FOR FAST DATA STORE?
- 27:12HPE FAST DATA STORES ENSURE COMPUTE RESOURCES ARE NEVE WAITING FOR DATA
related talks
24mManipulating Data in an Artificial Intelligence World
46mKeynote: When Hardware and Databases Collide
14mSoftware 2.0 Needs Data 2.0: A New Way of Storing and Managing Data for Efficient Deep Learning
18mThe "New Normal" or An Evolution
33mBuilding smarter ML and AI models and making them more accurate
31mScaling Autonomous-Driving R&D: A 100 PB, 100 Billion-File Open-Source Elastic File...
05h 34mData @Scale 2024
14mFrom Reservation to 1,600 GPUs: A Petabyte-Scale Ray Data Pipeline