Pains and Gains: Lessons From Orchestrating Volumes for Remote Storage in Cloud Native AI Scenarios
Talkat Data on Kubernetes Day Europe 2025 · on Tuesday, 2025-04-01· at 10:40· Level 1 | Hall Entrance N10 | Room E·
summary
Orchestrating remote object/NAS volumes uses CSI + FUSE and this causes extra CPU/memory on nodes, dependency conflicts between FUSE builds, unclear host-side logs, and residual broken mounts after remounts. The solution runs containerized FUSE clients with a mounter daemon that uses a premount with FD technique so CSI opens the mount and FUSE reuses the file descriptor; this avoids leftover mounts and enables pod-level auth. Fluid adds dataset and runtime CRs to create distributed cache workers; demo shows model load latency improves about 5× for an 8B Llama and FUSE crashes recover without restarting the app.
AI-written summary · about
chapters (13)
- 0:00AI storage challenges
- 1:48Remote storage scenarios
- 4:58Problems with FUSE clients
- 7:27Containerized FUSE clients
- 10:01Remote storage stability
- 12:30Mount recovery operator
- 14:58Premounting for recovery
- 15:19Demo: FUSE client crashes
- 20:15Fluid for dev environments
- 21:20Fluid dataset concepts
- 23:47Distributed cache setup
- 24:50Caching for AI serving
- 25:54Q&A
related talks
20mAdvanced CSI-FUSE Filesystem for AI/ML Data Management in Kubernetes
22mCSI Driver X FUSE Drivers: a Kubernetes Object Storage Solution for AI/ML Data Portability
26mEnabling Seamless AI Workloads: Achieving Zero-Downtime Upgrades for FUSE in Kubernetes
37mBuild a Kubernetes Based Cloud Native Storage Solution From Scratch
25mStreamlined Efficiency: Unshackling Kubernetes Image Volumes for Rapid A...
29mSo, What If I Don’t Want My Persistent Storage To Be Yet Another Bindmount?
08mGold Sponsor Lightning Talk: The Future of Container Storage: CSI and Cloud Foundry
27mThe Storage Architecture of Intel's Data Management Platform (DMP)