Lightning Talk: Ramalama - The Project That Brings AI and Containers Together While Making AI Boring
summary
Ramalama is a CLI that wraps Podman to run LLMs and vision models as OCI containers, auto-detecting GPUs (CUDA, ROCm, Vulkan, Intel, Ascend) and pulling GPU-specific images so no manual recompilation is needed. It runs rootless containers by default and isolates network access, supports Hugging Face and OCI registries, and converts model formats and images. It generates Kubernetes YAML and systemd quallet units for production and demos a camera app and HTTP API on a ThinkPad T480X CPU. ramalama run and ramalama pull work end-to-end.
AI-written summary · about
chapters (13)
- 0:00Ramalama project intro
- 0:43AI and containers combined
- 1:27Run models locally
- 2:10Developer benefits
- 3:12Ramalama with Podman
- 3:53GPU optimization
- 4:57Model selection simplified
- 5:40Security and isolation
- 6:22Deploy models to production
- 6:43Live demo
- 9:54Production configs
- 10:37Multimodel apps demo
- 11:20Q&A
related talks
37mWhat's new with RamaLama
35mRamaLama: Running AI Models in Containers.
47mRamaLama: Making working with AI Models Boring
32mGet Up and Running with Llamastack to Create AI Applications!
17mUnpacking the Llama Stack: Architecting Next-Gen AI Applications
07mKeynote: Accelerating GenAI Adoption with Containers: Unleashing AI/ML Development...
29mPodman AI Lab: For developers to build AI Applications with LLMs running locally 
40mRolling out Enterprise AI: Tools, Insights, & Team Empowerment