From Generalist to Specialist: Customizing LLMs for Building Laws in Norway
summary
Norkart builds an LLM-based RAG pipeline to answer "Can I build a garage here?" for ~90,000 annual Norwegian building applications; 2022 audits show ~40% contain errors. Improvements focus on chunking: research finds semantic independence of chunks correlates with RAG performance, so a custom chapter chunker uses an LLM to detect headers, output JSON and include headers in chunks. Add query expansion for legal terms; system reaches ~97% coverage on benchmarked questions and surfaces source chunks.
AI-written summary · about
chapters (20)
- 0:00Customizing LLMs for Norwegian laws
- 2:59Building application challenges
- 8:00LLMs and zonal plan complexity
- 9:49RAG systems workflow
- 13:46RAG limitations and vector DBs
- 14:32Chunking methods: fixed, recursive, semantic
- 18:44Measuring chunk quality
- 24:04Chapter-based chunking fixes
- 27:34Query expansion for legal terms
- 30:59Prompts for legal accuracy
- 33:06Legal principles in LLM outputs
- 35:13Zone data in prompts
- 36:39References and UI clarity
- 39:05PLUMBRAT beta testing
- 39:26Datasets and RAG benchmarks
- 41:54Why RAG benchmarks fail
- 44:50Metrics for legal RAG accuracy
- 45:13AI agents and MCP servers
- 48:27RAG vs full-document prompting
- 51:19Q&A
related talks
36mTechniques to avoid common LLM pitfalls
52mAnatomy of an LLM
01h 34mHow to Construct and Assess Your First LLM Based RAG System
01h 01mLLMs in Action: How Norkart Leverages AI
52mGive Your LLMs a Left Brain
28mThe 10 Commandments for Crafting Custom LLMs with Domain Mastery
33mDesigning LLM Applications for Massive Datasets
22mPractical Strategies for Implementing Domain-Specific LLMs