Build your enterprise Generative AI applications with Microsoft Olive
summary
Demo shows Microsoft Olive as an open-source SLM pipeline that runs fine-tuning with QLoRA/LoRA, converts models to ONNX / GGUF and quantized formats (FP16, INT4), and pushes jobs to Azure Machine Learning or local GPU/DirectML. The session runs Phi-3/Phi-2 variants via Semantic Kernel, LlamaEdge and WasmEdge, and successfully runs a quantized Phi-3-mini on an NVIDIA Jetson Xavier NX (Jetpack 5.01) with low memory and good latency.
AI-written summary · about
chapters (10)
- 0:00Introduction to the session
- 1:21Understanding SLMs
- 2:34Microsoft Phi series models
- 9:44Demo: Deploying SLMs
- 12:55Hybrid AI future
- 15:05Enterprise SLM strategy
- 23:31Microsoft Olive explained
- 28:43Integrated AI workflow
- 31:04Advanced demo and evaluation
- 34:23Edge performance and closing
related talks
46mAI made easier: How the ONNX Runtime and Olive toolchain will help you Q&A
45mDeliver AI-powered experiences across cloud and edge with Windows
23mFoundry Local: Cutting-Edge AI experiences on device with ONNX Runtime/Olive
52mSimplifying the adoption of generative AI for enterprises
47mOwn Your LLM: Running Generative and Agentic AI Locally
38mIntroducing Open Platform for Enterprise AI
21mDemocratizing AI with Open Source Multi-Agent Systems
11mNVIDIA and the AI-Powered Enterprise