Real-Time Data Ingestions for Data Fabric and AI Applications
summary
Oracle GoldenGate claims real‑time ingestion into Apache Iceberg with two replication designs including engineless iceberg ingestion that writes parquet files plus metadata and does a configurable 15 minutes aggregation microbatch before commit. Targets include Snowflake BigQuery Databricks or REST Glue Nessie catalogs and an async API produces native JSON. Customers prove it end to end: a bank feeds IBM Watson X and an airline replaces nightly batch with near‑real‑time partner feeds. A free Golden Gate container exists for testing.
AI-written summary · about
chapters (20)
- 0:00Introduction
- 1:08Oracle GoldenGate overview
- 3:56GoldenGate’s evolution
- 6:27Key use cases
- 7:33Apache Iceberg intro
- 8:38Engineless Iceberg design
- 10:24Managed Iceberg tables
- 11:49IBM Watson X use case
- 12:54American Airlines pipeline
- 15:41Batch to real-time shift
- 16:46Recovery architectures
- 18:11Data mapping challenges
- 20:00Table replication filtering
- 21:25Q&A
- 22:51Real-time challenges
- 25:06Free GoldenGate edition
- 25:49Databricks user access
- 26:34End-to-end replication
- 28:01Replication orchestration
- 29:07Synapse bottlenecks
related talks
13mReal-time Data for AI and RAG with GoldenGate 23ai
30mOCI GoldenGate: Elastic, real-time data integration
27mDistributed AI Processing with GoldenGate 23ai
20mOracle stream analytics on data in motion
36mProductizing AsyncAPI for Data Replication
38mOracle GoldenGate roadmap and strategy
31mSuperfast event stream processing with a converged database
26mArchitecting real-time transactional data lake