Internal AI solutions -from hype to production
summary
Pensionsmyndigheten runs open-source LLMs and niche ML inside its internal Java microservice stack: Llama on CPU packaged in containers, integrated via Spring AI and artifactory, behind a custom LLM portal that enforces auth, throttling and async jobs. 8‑CPU/80GB nodes process a 4,000‑char comment in ~15 minutes so use cases run as batch. A handwritten‑digit model is fine‑tuned via PyTorch/Transformers from Hugging Face, improving from ~10% to 98% accuracy on generated test data for ~15,000 forms, then deployed with Deep Java Library.
AI-written summary · about
chapters (7)
- 0:00Internal AI solutions journey
- 3:59Current infrastructure
- 8:17LLMs on CPUs case study
- 19:57Niche ML models case study
- 28:35AI limitations and challenges
- 36:29AI adoption learnings
- 41:09Q&A
related talks
49mTame your Llama, run AI in Java
50mGenerative AI in Practice: Concrete LLM Use Cases in Java with the PaLM API
46mThe Era of AAP: Ai Augmented Programming using Modern Java
01h 01mScaling AI with Java: Building Production-Ready LLM Applications
57mPure Java Enterprise AI/LLM Integration (EAI 2.0)
27mAI Demystified Unveiling the Intelligence Behind the Machine
32mScaling Generative AI: Building Production-Ready LLM Applications
46mAI-powered Java Developers