RamaLama: Running AI Models in Containers.
summary
RamaLama is a developer tool that containerizes and runs local AI models: it wraps Podman/Docker, auto-detects GPUs and pulls GPU‑optimized OCI images (CUDA, Vulkan, ROCm, Intel), supports cpu fallback, and defaults to llama.cpp with optional VLM and whisper.cpp. Containers run rootless, with dropped capabilities and network=none by default. It converts Hugging Face models into OCI images, serves an open AI REST API, generates Kubernetes YAML/quadlets, and ships weekly (v0.9.1, ~100+ contributors).
AI-written summary · about
chapters (6)
- 0:00RamaLama project intro
- 3:34Run AI models locally
- 7:52Architecture and goals
- 13:28Commands and workflow
- 21:36Live demo and security
- 26:57Q&A
related talks
47mRamaLama: Making working with AI Models Boring
37mWhat's new with RamaLama
12mLightning Talk: Ramalama - The Project That Brings AI and Containers Together While Making AI Boring
29mPodman AI Lab: For developers to build AI Applications with LLMs running locally 
17mUnpacking the Llama Stack: Architecting Next-Gen AI Applications
37mAI Lab Recipes: Cooking up AI Applications on your Laptop
45mEverything You Need to Know About Running LLMs Locally
31mDeploying Large Language Models with Ease