Operational analytics with AWS
summary
Amazon OpenSearch Service claims to ingest and index high‑velocity, disparate machine data for interactive search and aggregations, supports 3 petabytes per cluster, and maintains compatibility with 19 Elasticsearch versions. It advertises up to 90% cost reduction using ultra‑warm/cold tiers, built‑in TLS/KMS encryption and multi‑AZ replicas, and offers S3 event + Lambda bulk API or Kinesis + Lambda buffer pattern for ingestion. Popular collectors include fluentd, fluent‑bit and Logstash, and aggregation uses OpenSearch Dashboards and vector similarity search.
AI-written summary · about
chapters (14)
- 0:00Start
- 4:08Machine-generated data is exploding
- 5:26Machine data contains valuable operational insights
- 7:17Elasticsearch is great for operational analytics
- 10:34Search and analytics
- 12:35Benefits of Amazon ES
- 17:39Shards are primary or replica
- 18:21Shard distribution on data nodes
- 20:13Scaling is important for Operational data
- 22:28Amazon S3 event notifications approach
- 24:15Amazon Kinesis approach
- 25:01Collectors: Popular options
- 25:44Buffers: Popular options AWS native
- 27:51Learn data analytics with AWS Training and Certification
related talks
35mFrom Chaos To Clarity: Centralizing Observability With OpenSearch at Scale
38mMigrating from Self-Managed ElasticSearch to OpenSearch
52mOperating OpenSearch at Scale: Lessons From Managing 10,000+ Clusters Across...
24mClosing Remarks & Mastering Efficiency: Pathbreaking OpenSearch Index Management (Ingestion, Index)
32mUnleash Your Cluster's Potential with OpenSearch Benchmark
46mOperating an OpenSearch Powered E-commerce Search Engine at Scale
39mOperating OpenSearch at Scale With Kubernetes Operators: Lessons Learned and Community Insights
21mNoOps for OpenSearch: The Future of Self-Healing Search Infras...