Writes, 3 ways: Postgres, Apache Kafka® and Apache Iceberg™
summary
Compares how writes actually happen: Postgres write-ahead log records every SQL change and enforces atomic, disk-first transactions so correctness beats raw speed; Kafka partitions and leader brokers run on a minimum three-node cluster with producer-assigned partitions, replicated followers and an ack-based producer flow that prioritizes replication and failover over strict read consistency; Iceberg metadata catalog is a JSON/YAML layer that points to Parquet/ORC files in S3, providing a table-shaped abstraction that bakes separation of storage and compute and enables semantic metadata work like the Open Semantic Interchange.
AI-written summary · about
chapters (7)
- 0:00Introduction and background
- 1:12Learning Apache Iceberg quickly
- 2:35Timeline of data systems evolution
- 4:00Writes in Postgres explained
- 8:53Writes in Kafka architecture
- 13:05Iceberg as metadata abstraction
- 17:57Future of data with AI and metadata
related talks
34mApache Iceberg.
01h 12mHands-on with Apache Iceberg
41mTip of the Iceberg
32mDiscover the Modern Way to Build Real-Time Analytics Solutions at Any Scale
50mDesigning a scalable data platform based on Apache Spark/Iceberg
05mMaking Iceberg walk and talk like Postgres
37mApache Iceberg: An Architectural Look Under the Covers
39mKafka as a Platform: The Ecosystem from the Ground Up