Store AI/ML models efficiently with OCI Artifacts
summary
Claim: OCI Artifacts store, version and distribute AI models as content-addressed blobs so updates send only differences. Uses the ORAS CLI and the mldm Python tool to push OCI manifests of files, breaks models into tensor-level blobs via model sharding, and reassembles them on-device. Kubernetes can mount OCI artifacts as volumes (new alpha), shared caching avoids duplicate downloads across namespaces, and a 50 GB model typically shards into the lower hundreds of files while very large models approach ~700 GB.
AI-written summary · about
chapters (9)
- 0:00Introduction
- 2:55Model storage challenges
- 10:12Why storage tools fall short
- 15:38Production deployment needs
- 18:30OCI Artifacts for models
- 24:31Model sharding beyond blobs
- 27:33Q&A: Model updates & sharding
- 31:00Q&A: Layered serving approaches
- 34:14Future outlook & wrap-up
related talks
33mAI Model Distribution Challenges and Best Practices
21mSponsored Demo: Cloud Native AI: Model Management with Harbor & Velero
36mModifying the Immutable: Attaching Artifacts to OCI Images
23mPanel: Expanding Argo CD Content Management Using OCI Artifacts
29mNavigating Innovation with Open Hybrid Cloud and Openshift AI
14mAI Deployment: Mastering LLMs with KFServing in Kubernetes
45mOCIv2: Container Images Considered Harmful
35mRamaLama: Running AI Models in Containers.