Lightning Talk: Hybrid LLM Inference With LangChain and Agent2Agent
Lightningat Open Source Summit Japan + AI_dev: Open Source GenAI & ML Summit + Automotive Linux Summit 2025 · 2025-12-08..12-10·
summary
A LangChain-based agent orchestration ties specialized agents to either cloud LLMs or on‑prem open models, with Kubernetes GPU hosts running models via KServe / llm-d that expose an OpenAI‑compatible API so any agent can call either backend. Using Azure OpenAI, Amazon Bedrock and OpenAI API for advanced, low‑volume steps and on‑prem models for high‑volume or sensitive work cuts inference cost versus cloud‑only multi‑agent setups that raise token usage “several times” per incident, but output quality and per‑task model selection remain unresolved.
AI-written summary · about
related talks
10mLightning Talk: LLM-Driven Applications: Intro to LLM Agents in Kubernetes
36mLLMs and AI agents demystified
01h 43mBuilding LLM Agents (Hackathon)
08mKeynote: Plug in and Scale: Serving LLM Models on Kubernetes Ma...
34mAI Agents: Transforming Autonomous Decision-Making
41mBuild Next-Generation Applications with Language Models using LangChain and Python
25mLangChain with Azure AI: Building LLM powered Applications
33mScalable LLM Inference on Kubernetes With NVIDIA NIMS, LangChain, Milvus and Flu...