Easy logging refinement with FlowG
summary
FlowG is a low‑code, flow‑based log pipeline that parses, classifies and forwards logs using Vector Remap Language, accepts HTTP/JSON, syslog and OpenTelemetry inputs and writes to a local store backed by badgerdb or forwards to Elasticsearch, Datadog, Splunk and others. Deployed across dozens of clusters at the European Commission, it stores per‑cluster one‑day logs, filters out debug/info before central forwarding, anonymizes IPs to cut storage and bandwidth, and supports a visual editor, REST import/export and an experimental Kubernetes chart; clustering replication exists but is untested.
AI-written summary · about
chapters (7)
- 0:00Introduction
- 1:20What is logging
- 10:10Why FlowG exists
- 15:27FlowG features
- 20:33FlowG in production
- 24:15Performance and future
- 26:10Q&A
related talks
39mMaking Logs Work for you with Fluentd
31mThe Long and Winding Road to Application Logging [A]
46mBuilding a scalable logging platform
21mLogging at scale using Graylog - Billion+ messages, 100K req/sec
22mFluent Bit - Swiss Army Tool of Observability Data Ingestion
28mLogfire - Uncomplicated Observability
41mNetwork Observability
49mRelational Observability for Cloud-Native Security and Data Science