Scaling Welfare With Open Source: How NAV Migrated To Managed OpenSearch
summary
Norway’s NAV migrates a 60TB logging footprint from on‑prem Elastic to managed OpenSearch with a zero‑downtime dual‑write migration, moving hundreds of GB/day of logs and traces into Aiven’s platform. Architecture uses Terraform, a management plane that autoscaled nodes, VPN tunnels for cluster traffic, Data Prepper for traces, and automated snapshots; the result runs on ~15‑node cluster with fewer resources than before and meets production load.
AI-written summary · about
chapters (20)
- 0:00Introduction
- 2:07NAV and Nordic welfare model
- 3:33Project to product shift
- 6:24Open source platform building
- 8:12Infrastructure modernization
- 10:00Managed OpenSearch vs Kafka
- 10:42Ivan’s managed services role
- 12:52Logging and observability challenges
- 15:20Open source first strategy
- 16:27Managed OpenSearch service
- 21:52Live OpenSearch cluster demo
- 23:35OpenSearch node roles and features
- 24:59OpenSearch challenges
- 26:50Elasticsearch to OpenSearch migration
- 31:28APM tools during migration
- 33:37OpenSearch 3.x upgrade issues
- 36:45Migration lessons learned
- 38:12Migration outcomes and benefits
- 39:35Future plans and contributions
- 40:17Q&A
related talks
35mFrom Chaos To Clarity: Centralizing Observability With OpenSearch at Scale
38mMigrating from Self-Managed ElasticSearch to OpenSearch
36mScaling Log Management: PB-Scale Migration From Elasticsearch To...
11mPractical Migration From Distributed OpenSearch Environments To Enterprise-wide SIEM Platform
35mInnovating at Scale: Lessons from Pegasystems' OpenSearch Journey
52mOperating OpenSearch at Scale: Lessons From Managing 10,000+ Clusters Across...
41mScaling Observability: From Chaos to Clarity with Open Source Tools
21mNoOps for OpenSearch: The Future of Self-Healing Search Infras...