Implementing CDC with change feed
summary
Shows how to implement CDC using Cosmos DB change feed pull to build a streaming ETL: a reader app pages changes, publishes events to Azure Event Hubs, and two writers update MongoDB and Azure Cache. Checkpointing uses the change feed continuation token so the pipeline resumes after crashes (demo stops, 27–40 inserts are replayed). Ordering is guaranteed per logical partition, deletes aren’t emitted, and the feed returns full items (no operation type), so dedupe or soft‑delete logic is required.
AI-written summary · about
chapters (17)
- 0:00Introduction
- 0:50Agenda
- 1:31What is CDC
- 5:30CDC requirements
- 7:29What is change feed
- 9:30Querying change feed
- 10:51What is continuation token
- 11:53Implementing CDC flow
- 13:27Considerations
- 14:20Solution
- 19:16Demo applications
- 20:01Start source application
- 22:07Insert new items
- 24:01Demo
- 26:50Code
- 30:32Reliability scalability
- 32:40Limitations
related talks
05mReal-world examples of real-time applications using change feed
36mEvent-Driven Change Data Capture pattern
24mCDC: From Zero to Hero
39mBuilding Event-Driven Systems at Scale with Cosmos DB
30mChange Data Streaming Patterns in Distributed Systems
01mA Change-Data-Capture use-case: designing an evergreen cache
14mHow not(!) to build real-time apps
41mStreaming your database - Easier said than done?