Lightning Talk: How to Bring Data Locality to I/O-Intensive Workloads on Kubernetes

on YouTube ↗05m

" summary

Alluxio introduces a virtualization layer between compute and storage to serve data without copying, claiming measurable I/O gains by bringing metadata locality and data locality into Kubernetes clusters. It mounts via volume/FUSE-like interfaces and offers Helm chart and operators for cloud or on‑prem deployment. The demoed system coordinates workers to prefer local data paths, targets faster data staging for analytics and large‑scale AI training, and reports reduced transfer and management overhead.

AI-written summary · about

" related talks

• 45mEmbracing Big Data Workload in Cloud-Native Environment with Data LocalitySammi Chen, Xiaoyu Yao · KubeCon + CloudNativeCon China 2019
• 45mAlluxio Data Orchestration for Machine LearningLu Qiu, Bin Fan · ApacheCon@Home 2021
• 29mOn Data Locality in KubernetesChen Wang, Shouwei Chen · Kubernetes AI Day North America 2022
• 34mOn Data Locality in KubernetesChen Wang, Shouwei Chen · ContainerCon @ Open Source Summit Japan 2022
• 25mOvercoming I/O Bottlenecks in LLM Training with Open-Source Distributed...Lu Qiu, Jasmine Wang · Open AI + Data Forum @ Open Source Summit Europe 2023
• 06m⚡ Lightning Talk: Colocating Data Workloads and Web Services on Kubernetes to Improve Resource UtilizationHe Cao · Data on Kubernetes Day North America 2023
• 40mOpen Source Memory Speed Virtual Distributed Storage SystemGene Pang · Data By the Bay 2016
• 45mEmbracing Big Data Workload in Cloud-Native Environment with Data LocalitySammi Chen, Xiaoyu Yao · KubeCon + CloudNativeCon + Open Source Summit China 2019