Growing to full-suite observability with Elasticsearch & the ELK Stack (PRT267)

on YouTube ↗52m

" summary

Dish Media describes moving from many point tools to a single Elastic stack pipeline that ingests 10 billion records a day from 25 million endpoints, uses Elastic Agent and Fleet for collection, and adds APM and OpenTelemetry for traces and metrics. The stack runs machine‑learning jobs for anomaly detection and forecasting, surfaces business KPIs alongside operational metrics, cuts investigation time from weeks to hours, and during a hurricane automatically detects signal loss and reroutes satellites to restore about 80% of affected customers.

AI-written summary · about

" chapters (22)

1. 0:00Intro
2. 3:11The journey to full-suite observability
3. 3:56We live in a world of exponential data growth and increasing complexity
4. 4:52Advertising data lifecycle
5. 7:26The everyday story before observability
6. 9:44A solution that...
7. 12:51Understanding operational and business KPIs
8. 13:31Requiring monitoring across hybrid cloud
9. 16:59Centralizing all data
10. 18:36Ingest pipeline
11. 20:11Elastic Agent: The single unified agent for logs, metrics, security, and more
12. 21:34Collecting logs, events, and metrics
13. 22:06Elastic APM: Flexible options for tracing
14. 24:01Enrich the data for insights
15. 25:52Benefits of full observability for insights
16. 31:32Leverage comprehensive machine learning
17. 31:56Go from reactive to proactive
18. 35:45Machine learning applied to business data
19. 37:29Observability and security: Start from development, not production
20. 38:56The Dish dev cycle: Or do you test in prod?
21. 41:31Security monitoring
22. 45:56Before you go, here are tips to take with you

" related talks

• 36mObservability from 0 to 100Nico Krijnen · ContainerDays 2022
• 19mScale observability and secure data in motion with Elastic and ConfluentJennifer Garcia, Anand Venugopal · Microsoft Build 2023
• 37mBuilding Modern Observability Architecture With OpenSearch, OS Ingest...Luis Tiani, Lucas Vieira · OpenSearchCon North America 2025
• 40mDeploying OpenTelemetry Organizationally: From Proof of Concept to In-Production at ScaleBharathi Ramachandran · Dash 2022
• 34mMonitor Kubernetes cluster with ElasticAshish Tiwari · KCD Mumbai 2023
• 01h 00mElasticsearch: you know, for more than searchRuss Cam · NDC Sydney 2017
• 10mEffective Observability: Best Practices with ElasticEvelien Schellekens · Voxxed Days Brussels 2023
• 01h 02mDesigning an open-source observability stack for KubernetesAsaf Yigal · AWS re:Invent 2019