Stream from multiple on-premises sources onto AWS
summary
HVR streams on‑prem databases to AWS using log-based change data capture, reading recovery/redo logs to capture every committed change and preserve commit order via commit timestamps. Targets include Redshift, RDS/Postgres, Snowflake and S3; supports Oracle, SQL Server, Postgres, DB2, iSeries and mainframes. The system does initial loads plus ongoing compressed, encrypted transfers, offers data validation and SAP cluster/table decoding, exposes latency and volume monitoring in a hub control UI, and runs on a hub-and-spoke distributed architecture with optional bidirectional replication and lightweight transforms.
AI-written summary · about
chapters (15)
- 0:00Intro
- 0:15Agenda
- 0:47Whats the best approach to get data onto AWS
- 3:32Maintaining consistency
- 4:20Commit sequence
- 4:39Commit time stamps
- 5:16Change data capture
- 6:32Cost
- 8:17Logbase CDC
- 9:19HBR on AWS
- 11:31Complex Enterprise Data
- 14:53Security Concerns
- 17:18Demo
- 18:54Charts
- 19:52Questions
related talks
37mBuild a Modern Data Platform on AWS
26mArchitecting real-time transactional data lake
29mGrowing insights through hybrid data architectures
39mUsing Streaming Data Pipelines in the Cloud to Achieve Success
45mModern Data Architecture
42mTransform your data approach: Develop a modern data strategy
53mDefine data streaming architectures & solutions to fit your needs
43mChange data capture in production with Apache Flink