Hive-Iceberg - Breaking the Ice: A Closer Look at Hive-Iceberg
summary
Hive 4.0 adds native Iceberg support so Hive queries use Iceberg tables via stored by Iceberg and a Hive Iceberg storage handler that reads/writes Parquet/ORC; metadata lives in HMS, stats are stored in Puffin for the CBO, and DDL/DML just works. New features include branching and time travel with snapshots, tag/expire/rollback, and an alter table ... convert to command to migrate external tables without rewriting data. Measured results show up to 70% faster on 1 TB TPC-DS (99 queries) in cloud and select tests where merge-on-read is 11s vs copy-on-write 35s; demos run from the Hive Docker image.
AI-written summary · about
chapters (15)
- 0:00Introduction to Hive-Iceberg
- 1:52Background on Hive and Iceberg
- 4:07Iceberg as next-gen table format
- 5:14Storage handler integration
- 6:19DDL and DML with Iceberg
- 7:24Branching and tagging features
- 8:46Snapshot management in Iceberg
- 10:12Statistics and query planning
- 11:55Merge-on-read vs copy-on-write
- 14:01Compaction strategies
- 15:05Migration to Iceberg tables
- 16:30Authentication and authorization
- 19:00Performance improvements
- 21:30Live demo of Iceberg features
- 26:12Closing and Q&A
related talks
34mApache Iceberg.
39mIntroducing Apache Iceberg: Tables Designed for Object Stores
31mHow Apache Iceberg Enables Multi-Engine Data Platforms
41mTip of the Iceberg
49mHow Apache Iceberg Really Works
50mDesigning a scalable data platform based on Apache Spark/Iceberg
52mApache Iceberg - An Architectural Look Under the Covers
13mExploring the Depths of Apache Iceberg's Metadata Capabilities