Operating OpenSearch at Scale With Kubernetes Operators: Lessons Learned and Community Insights
summary
Runs OpenSearch at scale for 17,000 instances using a homegrown Kubernetes operator that models flexible node groups and uses StatefulSets with operator-controlled rolling updates to enforce different upgrade orders (config changes: cluster-managers first; major-version upgrades: data nodes first). Adds an OpenSearch status reader that polls every ~15s to update CR conditions and triggers reconciles only on change, avoiding constant controller load. Open-source operator v3.0 alpha adds smart scaler (safe shard drain), online volume expansion, TLS and plugin management and reports broad community adoption.
AI-written summary · about
chapters (15)
- 0:00OpenSearch Kubernetes Operators intro
- 1:47Designing OpenSearch Operators
- 5:25Node groups for flexible topologies
- 7:51Operator reconciliation and config
- 8:12OpenSearch version upgrade challenges
- 10:19Shard recovery during updates
- 12:27StatefulSets and rolling updates
- 14:37Monitoring operator decisions
- 19:57Operator challenges and solutions
- 20:18OpenSearch Operator overview
- 21:21Operator features and capabilities
- 26:00Operator 3.0 alpha highlights
- 27:02How to contribute to Operator
- 28:46Community and contributors
- 36:04Q&A on Operator features
related talks
23mCreating the OpenSearch Kubernetes Operator
52mOperating OpenSearch at Scale: Lessons From Managing 10,000+ Clusters Across...
40mOperating Opensearch - The Kubernetes Way
36mAutoscaling Search and Analytics in Kubernetes: The OpenSearch Operator Advantage
42mOpenSearch in Kubernetes: Helm Charts and Operator
14mKeynote: OpenSearch on Kubernetes at Atlassian
21mNoOps for OpenSearch: The Future of Self-Healing Search Infras...
32mAutoscaling Elasticsearch for Logs on Kubernetes