Elasticsearch Data Streams: Best Practices for Your Time-Series Data
summary
The talk explains Elasticsearch data streams for time-series data, covering index templates, ILM (Index Lifecycle Management), rollover, and aliases so Elasticsearch auto-creates time-series indices with correct mappings and settings and requires an @timestamp field. It describes the write/read model using a hidden current write index and reads across backing indices, recommends single-shard indices with rollovers around 50 GB per primary shard, notes Beats can provision templates and policies automatically, and mentions a Migration API to convert alias+rollover setups to data streams.
AI-written summary · about
chapters (8)
- 0:00Introduction
- 6:59Temporal data indexing
- 13:24Mapping best practices
- 21:10Alias management and reindexing
- 28:08Shard sizing and scalability
- 32:06Data Streams introduction
- 39:34Data Streams usage and benefits
- 42:42Q&A
related talks
27mLet’s Talk About Time - Data Driven Organic Monitoring
42mContemporary Approaches to Data at Scale
48mLevel up your Big Data game with Redis Time Series!
32mAutoscaling Elasticsearch for Logs on Kubernetes
44mIntroduction to Time Series
37mBeyond the basics with Elasticsearch
42mBuilding a Streaming (E-health) Data Pipeline: When and How?
30mBuilding a Time-Series Database for Point-in-Time Lookups