Network Configurations for High-Performance Streaming Connectivity in K8s

on YouTube ↗30m

" summary

Hands-on comparison of Kafka/Redpanda networking in Kubernetes using sockperf on a three-node EKS cluster (m5.2xlarge): nodePort yields the lowest latency, load balancer is next, and adding a proxy such as Envoy proxy raises tail latency and halves efficiency at scale. Practical build details include StatefulSets with headless and per-broker services, advertised addresses via DNS or node IPs, NVMe local storage with XFS and LVM striping, CPU pinning for brokers, and tuning producers/consumers and partition counts to balance throughput, latency, and metadata overhead.

AI-written summary · about

" chapters (8)

1. 0:00Streaming on Kubernetes
2. 1:53Kafka vs HTTP services
3. 3:43K8s architecture for Kafka
4. 8:23NodePort vs LoadBalancer
5. 12:38Network options performance
6. 17:23Tuning Kafka brokers/clients
7. 23:05Producer/consumer configs
8. 26:34Q&A

" related talks

• 13mScaling Kafka on KubernetesShalini Katyayani Koney · Conf42 Kube Native 2025
• 34mDistributed applications and KubernetesMatthias Haeussler · ContainerDays 2022
• 27mRunning Kafka in Kubernetes a practical guideKatherine Stanley · Voxxed Days Microservices 2018
• 47mRunning Kafka in Kubernetes: a practical guideKatherine Stanley · Devoxx Belgium 2019
• 15mHow We Solved the K8s Cluster Communication ChallengeDavid Gerchikov · KubeDay Israel 2023
• 36mKafka Without Disks: Making Streaming Brokers Disposable on K8SAnton Borisov · ContainerDays Hamburg 2026
• 37mDisaster Recovery Options running Apache Kafka in KubernetesGeetha Anne · Strange Loop 2022
• 42mKubernetes Networking at ScaleLaurent Bernaille, Bowei Du · KubeCon + CloudNativeCon Europe 2019