take care of your logs
summary
Centralize logs instead of keeping them local by shipping via syslog to a collector, then parse and normalize using Logstash or the built-in processor and enrich IPs with MaxMind. Buffer bursts with Kafka or RabbitMQ, use Elasticsearch/Kibana for search and dashboards with Beats as shippers, and add alerting and active blocking (e.g., fail2ban) to detect correlated failures and support postmortems, ACLs and high-throughput scaling.
AI-written summary · about
chapters (13)
- 0:00Log management
- 3:40Logfiles importance
- 7:09Local log limits
- 13:13Centralizing with syslog
- 17:32Modern logs with Elasticsearch
- 23:58Normalizing log data
- 27:49Graylog processing
- 32:32Non-standard log formats
- 37:11Log transport and scaling
- 39:23Centralized logging benefits
- 44:29Security and compliance
- 46:41Reporting and analysis
- 51:11Q&A
related talks
30mCentralized Log Management
14mCentralized Logs using ELK (Elasticsearch, Logstash, Kibana) Stack
43mPractical Application Logging and Monitoring
39mFast log management for your infrastructure
01h 00mApplication Logging & Logstash
31mLogging at Scale, Billion+ messages, 100K msg/sec
46mBuilding a scalable logging platform
30mCentralized Logging Patterns