Announcing Delta Lake 4.0 with Liquid Clustering
summary
Delta Lake 4.0 ships a new layout and type system: Liquid Clustering replaces manual partitioning and reports up to 7x faster writes and 12x faster reads on benchmarks, with about 1,500 customers already using it and 1.2 zettabytes skipped. An open variant data type stores JSON efficiently (about 8x faster than raw strings), is implemented in Apache Spark and available in DVR 153, and is released as an open binary format for other engines.
AI-written summary · about
related talks
46mOpen Table Formats in the Wild: From Parquet to Delta Lake and Back
31mDelta Lake: Open Source Reliability and Quality for Data Lakes
39mDelta Lake: Diving into Data Lakes Without the Downsides
20mDelta Lake: Turn your Data Lake into a Lakehouse
38mGetting Started with Delta Lake
19mOpen Source Reliability for Data Lakes
28mLakehouse Architecture with Delta Lake and Databricks
58mBuild reliable data lakes with Delta Lake & Databricks