How Swisscom efficiently fine-tunes and deploys LLMs with Amazon SageMaker
summary
Swisscom fine-tunes a Llama 3.1 8B base using LoRA low‑rank adapters and supervised learning from synthetic prompt–completion pairs generated by a larger teacher model, stores anonymized data in S3, and runs repeatable SageMaker pipelines with model registry. Deployment uses CDK to provision SageMaker endpoints on G6e instances (currently two fixed nodes, ~90% GPU usage), serving ~50% CIVR traffic with median latency <250 ms and plans for dynamic scaling and continuous feedback.
AI-written summary · about
chapters (10)
- 0:00Introduction and speakers
- 1:26Swisscom’s customer service vision
- 9:20Choosing the right LLM
- 12:56Injecting business data into LLMs
- 16:54AWS services for LLM fine-tuning
- 17:36SageMaker model experimentation
- 20:48Building repeatable pipelines
- 22:59Deploying models with SageMaker
- 27:55Production results and impact
- 30:03Key takeaways and next steps
related talks
33mBuild, train, and deploy ML models using Amazon SageMaker
27mScaling ML to the next level: Hosting thousands of models on SageMaker
43mFrom Shadows to Spotlight - How Swiss Post Performs Reliable ML Deployment
19mUsing Amazon SageMaker to develop and deploy ML models
01h 00mHigh-performance & cost-effective model deployment with Amazon SageMaker
35mLLM Customization
19mDeveloping large-scale machine learning algorithms on Amazon SageMaker
34mFrom Shadows to Spotlight - How Swiss Post Performs Reliable ML Deployment