LeRobot Dataset v3.0: Scaling Reads with Ray
summary
LeRobot v3 data causes heavy redundant MP4 decodes because episodes span multiple video files and files contain many episodes; Ray Data implements read grouping by video file and lazy planning to cut opens from 4,600 to about 300 for the XVLA softfold 1,500‑episode set and reports ~45 average reopens per file in naive parallelism. The implementation decodes frames on worker pools, uses frame caching within episodes, and successfully scales to 100,000‑episode Droid sets with orders‑of‑magnitude fewer file opens.
AI-written summary · about
chapters (8)
- 0:00Introduction to LeRobot v3
- 1:10LeRobot Dataset v3 format
- 3:19Scaling reads with Ray Data
- 6:12Ray Data implementation
- 8:44Integration with Ray Train
- 10:10Key parameters and optimizations
- 11:54Case studies and results
- 13:22Takeaways and conclusion
related talks
35mHow Bytedance Builds Large-Scale Data Processing Pipelines for Multimodal Models with Ray
22mLarge Scale Data Loading and Data Preprocessing with Ray
24mBuilding Complex Data Analytics Pipelines with Ray
30mHow to Scale Unstructured Data Processing with Ray Data
32mAnyscale's Ray Data: Revolutionizing Batch Inference
26mData Processing on Ray
35mMars-on-Ray: Accelerating Large-scale Tensor and DataFrame Workloads
26mRayDP: Build Large-scale End-to-end Data Analytics and AI Pipelines Using Spark and Ray