Build a Cloud Native Logging Pipeline on the Edge with Fluentbit Operator
summary
Fluent Operator claims to manage Fluent Bit and Fluentd lifecycles in Kubernetes, enabling dynamic config reloading by generating configs into Secrets and using a watcher to reload without restarting pods. It supports three modes including fluent-bit + fluentd mode, uses Fluent Bit (C, ~70 plugins, 1B downloads) and Fluentd (1000 plugins), and was deployed on k3s/minikube with Kafka and Elasticsearch. The demo generates CRDs, mounts configs into DaemonSets, and verifies logs and new Elasticsearch indexes, showing the pipeline actually works.
AI-written summary · about
chapters (17)
- 0:00Start
- 2:03The Challenges of Logging in Kubernetes
- 3:47How to debug your workloads in Kubernetes
- 5:17Introduction to Fluent Bit and Fluentd
- 6:10Fluent Bit Logging Pipeline and Plugins
- 7:39Fluent Bit vs Fluentd
- 8:23Fluent Operator Intro - History
- 9:06Fluent Bit can not reload config gracefully
- 11:17Fluent Operator Intro - FluentBit Mgmt
- 12:38Fluent Operator Intro - FluentBit Operator
- 13:15Fluent Operator General Architecture
- 13:46Use Fluent Operator as you want
- 14:22Fluent Bit Only mode
- 15:14Fluentd only mode
- 16:36Fluent Bit + Fluentd mode
- 17:18Case Study: Fluent Operator in KubeSphere
- 18:05Demo: Collect K8s logs and send to ES
related talks
22mLogging: Fluentd & Fluent Bit
26mFrom FluentBit Operator to Fluent Operator: The Long Way to Come
34mPower of O11y with FLUENT BIT in AWS EKS environment
26mKubernetes and Logging: Do It Right
34mGetting Started With Logging in Kubernetes
12mLightning Talk: Achieving Unified Observability for Cloud and Edge with FluentBit
33mCloud Native Logging 101 [B]
28mBuilding Observability Pipeline with Fluent-Bit