Kafka 4, Fantastic?
summary
Kafka 4 introduces around 40 KIPs (18 in 4.1) that require real migrations: clients and Streams now need Java 11, brokers and Connect need Java 17, and old client/broker APIs and V0/V1 message formats are blocked for writes in favor of V2. ZooKeeper is removed and replaced by KRaft, moving broker metadata into an internal topic with a bridge-mode migration and rollback tools; the rebalance protocol reduces RPCs and eliminates stop-the-world pauses. Operational defaults change (linger.ms 0→5ms, max future timestamp 1h, min segment 1MB), and client/plugin metrics can stream to brokers via OTLP.
AI-written summary · about
chapters (9)
- 0:00Kafka 4 intro
- 3:14Migrating to Kafka 4
- 10:58ZooKeeper removal KRaft
- 16:56New metrics observability
- 20:48Consumer rebalancing
- 27:50Plugin improvements
- 30:22Kafka queues
- 35:15Connect plugin management
- 43:39Compatibility roadmap
