Spell of Transmutation - Using Debezium to Transfer Realtime Data Changes to Event Hub
summary
Shows how Debezium running in an Azure container reads SQL Server CDC and posts change events into Event Hub (using the Kafka endpoint), then downstreams to Azure Functions or Stream Analytics. Covers exact setup steps: enable CDC, open SQL firewalls, choose snapshot.mode (schema_only vs initial), provide a connector history store, and disable tombstones on delete. Warns that the connector continuously polls (so serverless SQL never sleeps), error logs surface only via the REST API, and examples on GitHub need careful tweaks.
AI-written summary · about
chapters (13)
- 0:00Introduction to Debezium
- 4:44Why use near real-time
- 6:46Understanding the components
- 11:34Getting started with Debezium
- 14:21Preparing the SQL database
- 18:48Setting up the event hub
- 23:50Deploying the container
- 27:25Configuring the connector
- 31:19Connector gotchas and tips
- 36:05Verifying and debugging
- 37:43Consuming the data stream
- 40:18Using stream analytics
- 43:15Conclusion and best practices
related talks
41mChange Data Streaming Patterns in Distributed Systems
01h 09mOpen-source Change Data Capture With Debezium
26mDebezium Server - New CDC runtime on the horizon!
23mChange Data Capture: A RabbitMQ and Debezium Approach
14mHow not(!) to build real-time apps
48mStreaming Database Changes with Debezium
42mData Streaming for Microservices using Debezium
49mChange Data Capture Pipelines with Debezium and Kafka Streams