Implementing Change Data Capture using Azure CosmosDB Change Feed mechanism
summary
Shows how Azure ComosDB SQL API change feed pull model reads pages of changes, pushes JSON items into Azure Event Hub and updates Redis cache and a Mongo replica, using change feed pull model + continuation token checkpointing to resume after crashes. Demo processes an initial 10-item snapshot then recovers and streams IDs 27–40 after a simulated outage, proving no data loss; limitations: full-item payloads only and deletes require soft-delete.
AI-written summary · about
chapters (15)
- 0:00Introduction
- 0:50Agenda
- 1:31What is Change Data Capture
- 5:30Typical CDC requirements
- 7:29What is a Change Feed
- 10:51What is continuation token
- 13:27Notes
- 14:20Solution
- 19:16Demo applications
- 20:01Start source application
- 22:07Insert new items
- 24:01Crash scenario
- 26:42Code overview
- 30:32Reliability and scalability
- 32:40Limitations
related talks
05mReal-world examples of real-time applications using change feed
39mBuilding Event-Driven Systems at Scale with Cosmos DB
01h 01mServerless & Schemaless with Azure Functions and CosmosDB
35mChange Data Capture: The Magic Wand We Forgot #bbuzz
21mBuild Scalable Inventory System with Azure Cosmos DB
31mChange Data Capture for non relational databases
01h 02mFrom SQL to Azure Cosmos DB
23mChange Data Capture: A RabbitMQ and Debezium Approach