Transitioning from Debezium ElasticSearch to Postgres for Streamlined Resilience
summary
Migrates a Debezium ElasticSearch pipeline into Postgres using logical replication as the chosen replacement, moving away from Kafka-backed Debezium and heavy ES clusters toward database consolidation where feasible. Describes prototypes with foreign data wrapper, measures cost and CVE burden from Java/Kafka/ES, and cites handling roughly 1.4 million systems per day. Shows reduced component count, fewer languages, explicit outage-isolation tradeoffs, and measurable cost and operational simplification.
AI-written summary · about
chapters (13)
- 0:00Introduction
- 1:10Transition purpose
- 2:57Use case overview
- 8:42Technical analysis
- 11:15Key outcomes
- 17:16Postgres alternatives
- 19:06Database consolidation
- 21:34Logical replication
- 24:06Foreign data wrapper
- 25:09Replication solution
- 25:52End-state architecture
- 28:22Implementation steps
- 30:11Q&A
related talks
51mGeo-Distributed Postgres Replication Using Debezium
42mWhy PostgreSQL people should care about Kafka and Debezium?
39mReliable Data Replication from Postgres on Kubernetes
48mBuilding Petabyte-Scale PostgreSQL Deployments
41mStreaming your database - Easier said than done?
27mBuilding Event-Driven Systems with PostgreSQL Logical Replication and Drasi
48mStreaming Database Changes with Debezium
41mChange Data Streaming Patterns in Distributed Systems