Presto Open Source SQL Engine for Massive Scale Data Lakehouses

on YouTube ↗51m

" summary

Presto is a disaggregated SQL engine that queries data in place (object storage, Hive metastore, RDBMS) with a coordinator + worker model that scales to thousands of nodes; Meta runs ~300 PB with ~1,000 concurrent users and Uber runs ~100 million ad hoc queries/day. Demo shows running the Ahana sandbox Docker on a laptop, starting coordinator and workers and using the UI to query S3/Hive tables. IBM surfaces Watson x.data, improvements to Iceberg connector, prestissimo native workers and planned performance gains (10x+ targets).

AI-written summary · about

" chapters (7)

1. 0:00Start
2. 1:32What is Presto?
3. 6:05Common Questions?
4. 11:25Scalable Architecture
5. 14:04Pluggable Presto Connectors
6. 15:56Presto Connector Data Model
7. 18:23Presto Hive Connector - Data File Types

" related talks

• 22mPresto for Your Fast, Open SQL LakehouseSteven Mih · OSA Con 2022
• 22mIntroduction to Presto: The SQL Engine for Data Platform TeamsRohan Pednekar, Philip Bell · Emerging OS Forum @ Open Source Summit North America 2022
• 50mPresto: SQL-on-Anything using KubernetesKarol Sobczak, Michael McCune, Matt Fuller · DevConf.CZ 2020
• 06mDriving enterprise open source adoption, from data lake to AIRon Bodkin · Strata + Hadoop World 2017
• 49mPresto in the CloudSivaramakrishnan Narayanan · Great Indian Developer Summit 2015
• 48mContinuous Optimization for distributed BigData analysisKai Sasaki · HighLoad++ Moscow 2018
• 34mBuilding Presto SQL experience on FlinkPeter Huang, Zhongting Hu · Flink Forward Global 2021
• 31mPanel: Open Data LakesDipti Borkar, Vinoth Chandar, Roy Hasson, Eric Kavanagh · OSA Con 2021