From Tower of Babel to Babel Fish: Evolving Kafka Architecture With Schema Registry
summary
Demonstrates a running Kafka cluster with Confluent Schema Registry in Docker connecting three services (orders, inventory, analytics) across Java, Python and Node.js, generates language-specific bindings from Avro schemas, adds fields order_timestamp and metadata and shows old consumers still read new messages (backward compatibility preserved) while an explicitly incompatible schema registration is rejected via the Registry REST API; metadata lives in a Kafka topic and consumers use schema ID lookup to fetch schemas, enabling Avro artifacts and CI/CD validation of contracts.
AI-written summary · about
chapters (8)
- 0:00Kafka polyglot reality
- 5:01Schema Registry as Babel Fish
- 12:08JSON/Java serialization limits
- 15:21Schema contracts alignment
- 20:43Schema Registry architecture
- 26:28Schema evolution rules
- 32:14Schema governance DevOps
- 34:41Q&A
related talks
01h 00mEvolving Your Kafka Architecture With Schema Registry
41mKafka as a Platform: The Ecosystem from the Ground Up
39mKafka as a Platform: The Ecosystem from the Ground Up
41mKafka as a Platform: the Ecosystem from the Ground Up
15mAn Empty Kafka is a Sad Kafka
38mStreaming with Structure
54mKafka as a Platform: the Ecosystem from the Ground Up
25mFrom Guesswork to Governance: Bringing API Discipline to Data Streaming