So You Want to Maintain a Reliable Event Driven System
summary
Five years ago a schema change causes a send loop that delivers 52 phishing emails; the concrete fixes and practices here are versioning topics (orders.orderconfirmed.v1), the outbox pattern for atomic state+event persistence, and a test harness that subscribes to local queues to assert events actually hit the message bus. Use the CloudEvents shape, emit event type+version in telemetry, prefer span links between domains and propagate inside a domain, and monitor queue depth, message age, inflight latency and error rates.
AI-written summary · about
chapters (9)
- 0:00Maintaining event driven systems
- 2:46Evolving event driven architectures
- 8:46Versioning events as APIs
- 15:29Testing async event systems
- 23:57Test harnesses for event verification
- 33:23Observability in distributed events
- 37:38Context in distributed tracing
- 47:49Upstream and downstream dependencies
- 51:43Key takeaways
related talks
49mSo You Want to Maintain a Reliable Event-Driven System
52mSo You Want to Build An Event Driven System?
52mSo You Want to Build An Event Driven System?
30mUnhappy Path & Dealing with Bad Events
54mSo You Want to Build An Event Driven System?
01h 13mChange Is Inevitable: Versioning Event-Driven Systems
59mThe Dark Side Of Events
31mThe Power of Event-Driven Systems without Burning your Hands or Budgets