Observability in a complex data migration: A system refactoring story
summary
A three‑phase migration replaces a legacy Kafka topic with a new event stream for a critical payment service while keeping downtime minimal and avoiding lost events. Phase 1 runs read-only consumption and traces plus a temporary datastore to validate data and test a converter, finding a couple of overlooked cases. Phase 2 adds logical ID deduplication and metrics; Phase 3 pauses, generates an overlap, swaps subscriptions, then watches processing vs consumption and business metrics and the dead‑letter queue until numbers match. Tools lean on OpenTelemetry/Grafana and it works within minutes.
AI-written summary · about
chapters (9)
- 0:00Introduction to observability
- 1:39Concepts and definitions
- 4:29Architecture and evolution
- 5:32Practical examples and tools
- 7:55The case for change
- 9:40Real-world migration context
- 12:21Strategy and risk assessment
- 16:12Implementing the strategy
- 23:51Final thoughts
related talks
27mObserving Failure in Reactive Systems
01h 02mAdding Observability to Distributed Systems
39mArchitectural design patterns applied to Distributed Systems
28mThings I wish I knew before we decided to migrate our metrics infrastructure..
31mCloud Migration Without Fear - A...
10mLightning Talk: Observability Migration Worst Case Scenarios
45mTracing: A journey to tactical insights
46mObservability in digital transformation