Sponsored Demo: Cloud Native AI: Model Management with Harbor & Velero
summary
Shows Harbor storing OCI AI artifacts using the CNAI spec and ORAS so a 1.1B-parameter TinyLlama (~600 MB) is pushed as cnai.model layers with deduplication, decoupling model from inference image. Kubernetes mounts model into a PVC via an init container and serves with llama.cpp on CPU. Velero plus the Copia filesystem plugin backs up namespace and PVC contents and restores them so the pod reuses cached model locally, returning faster than repulling from Harbor. oras push cnai.model and Velero restore are demonstrated working end-to-end.
AI-written summary · about
chapters (13)
- 0:00Cloud-native AI model management
- 1:46Traditional model packaging problems
- 3:09OCI artifacts for model storage
- 5:36Harbor as AI model registry
- 6:59Day-two operations with Velero
- 8:44Full architecture walkthrough
- 9:46Pushing model to Harbor
- 11:33Deploying AI models with Kubernetes
- 13:20Verifying model inference
- 14:45Backing up AI workloads
- 15:50Disaster recovery with Velero
- 17:14Summary
- 17:56Q&A
related talks
30mHarbor Maintainers Talk - Role of Harbor in AI, IOT Edge and the Future
35mStore AI/ML models efficiently with OCI Artifacts
25mCloud Native GenAI Using KServe and OPEA
35mRamaLama: Running AI Models in Containers.
47mRamaLama: Making working with AI Models Boring
14mAI Deployment: Mastering LLMs with KFServing in Kubernetes
45mWelcome to the AI jungle! Now what?
31mGotta Cache 'em All: Scaling AI Workloads With Model Caching in a Hybrid...