Podman AI Lab: For developers to build AI Applications with LLMs running locally
summary
Shows how Podman AI Lab (an extension of Podman Desktop) actually downloads curated models, runs inference servers with llama.cpp and Mistral, and wires them to a Python Flask microservice using LangChain so queries return structured JSON. Demo builds a search API that asks a model for an array of {name,URL}, runs models locally on a GPU‑enabled VM, and packages the app as a recipe so it can run LLMs locally with GPU-enabled VM and add custom recipe to AI Lab catalog.
AI-written summary · about
chapters (6)
- 0:00Introduction to Podman AI Lab
- 4:00Model experimentation
- 10:54Working with AI recipes
- 17:47Developing a custom search API
- 22:32Cataloging and deploying
- 27:38Conclusion and future steps
related talks
37mAI Lab Recipes: Cooking up AI Applications on your Laptop
59mBuilding #APIs in the #AI Era
35mRamaLama: Running AI Models in Containers.
32mGet Up and Running with Llamastack to Create AI Applications!
35mFrom InstructLab and Podman AI Lab to OpenShift AI: Simplifying LLM workflows
47mRamaLama: Making working with AI Models Boring
26mWelcome to the AI Jungle! Now What?
45mEverything You Need to Know About Running LLMs Locally