Get Up and Running with Llamastack to Create AI Applications!
summary
LlamaStack provides a unified API layer that connects model servers to apps; the demo runs llama 3 8B instruct (~5GB) served by Ramalama serve → Podman container on :8080 and a LlamaStack API on :8321, a Streamlit UI client, and a small Python chatbot — all containerized and working locally. Podman’s podify (podman generate/play) plus podman podify + quadlet → systemd unit auto-starts the stack. For production they show vLLM via vLLM/LLMD with prefill vs decode on miniKube (4 L4 GPUs), plus Prometheus/Grafana/Tempo tracing and RAG/tool integrations.
AI-written summary · about
chapters (11)
- 0:00Introduction to Llama Stack
- 1:48What is Llama Stack
- 2:51Setting up Llama Stack locally
- 8:34Running AI apps with Llama Stack
- 10:20Managing containers with Podman
- 16:41Transitioning to Kubernetes
- 20:10Understanding VLM and LLMD
- 23:25Demonstrating RAG and tools
- 24:48Monitoring in Llama Stack
- 28:03Future improvements
- 30:34Resources and closing
related talks
17mUnpacking the Llama Stack: Architecting Next-Gen AI Applications
47mRamaLama: Making working with AI Models Boring
45mEverything You Need to Know About Running LLMs Locally
37mWhat's new with RamaLama
29mPodman AI Lab: For developers to build AI Applications with LLMs running locally 
11mIntroducing Llama API: A Seamless Way to Revolutionize Your Workflow
11mBuilding easily deployable RAG applications with Llamastack
38mMeta's Roadmap for Full Stack AI