ModelPack: An Open Standard for Packaging, Distributing and Running LLMs...
Talkat Open Source Summit Japan + AI_dev: Open Source GenAI & ML Summit + Automotive Linux Summit 2025 · 2025-12-08..12-10·
summary
ModelPack defines an OCI-based model artifact format and a runtime/config schema, uses existing registries so Harbor (since 2.13) natively shows model metadata, and adds tools like modelctl CLI and a model CSI driver for Kubernetes. It integrates P2P distribution via dragonfly to avoid saturating networks (example: 70 GB Llama 3 on 100 nodes), supports layer mediaTypes for faster IO, shards large models (Deepseek R1 671 GB, >>1 TB) to cut startup time roughly fourfold when split across four nodes.
AI-written summary · about
chapters (7)
- 0:00ModelPack introduction
- 1:28Machine to model-centric infra
- 5:07Why ModelPack is needed
- 9:33ModelPack best practices
- 15:04P2P for bandwidth bottlenecks
- 18:57ModelPack beyond specs
- 26:12Summary and future
related talks
23mBringing Container-Native Simplicity to AI/ML
32mOperationalizing Openness: Standardizing AI Model Supply Chains With the Model Openness Framework
30mHarbor Maintainers Talk - Role of Harbor in AI, IOT Edge and the Future
44mFrom Containers To Cognitive Agents: Open Source Foundations for Enterprise-Grad...
15mKeynote: From Models to Market: What's the Missing Link in Scaling Open Source Models on...
19mOn-Demand Container Partitioning for Distributed ML
08mKeynote: Plug in and Scale: Serving LLM Models on Kubernetes Ma...
32mOrchestrating AI Models in Kubernetes: Deploying Ollama as a Nati...