Data @Scale Seattle- Eric Hwang introduced by Ganapathy Krishnamoorthy
summary
Presto runs directly on a sharded MySQL fleet to power user‑facing analytics, implementing three connector APIs (metadata, data location, data stream) and using InnoDB clustering and bucketed sharding to trade fan‑out for parallelism. Multiple physical layouts (separate MySQL tables) give alternate sort orders for constant‑memory aggregations; writes are eventually consistent and MySQL uses semi‑sync replication. Production deployment hosts several hundred terabytes, shows consistent low latency for interactive queries, and a native Raptor columnstore is in development.
AI-written summary · about
chapters (21)
- 0:00Start
- 4:30SEATTLE
- 4:50Presto
- 6:09Community Adopted NETFLIX
- 6:42Architecture
- 10:55Pluggable Storage
- 13:04Insight Platforms
- 13:57Requirements
- 15:49Why?
- 17:06Sharded MySQL
- 18:18Full Scan
- 19:07Targeted Scan
- 19:29Parallelism vs. Fanout
- 20:08Bucketed Sharding
- 21:10Bucketed Scan
- 22:04Clustered Data
- 25:07Clustered Query Execution
- 27:29Hot and Cold Data
- 28:19System tuning knobs
- 29:40What's Next?
- 30:25Big Picture
related talks
22mPresto for Your Fast, Open SQL Lakehouse
22mIntroduction to Presto: The SQL Engine for Data Platform Teams
46mPresto - Today and Beyond - The Open Source SQL Engine for Querying all Data Lakes
25mThe future of databases: Combining operational and analytical data at massive scale
55mThe Future of MySQL
27mAutomatically shard & scale-out traditional databases on K8s
37mScale MySQL beyond limits with ProxySQL
58mIntroducing ScaleDB, the New Clustered, Distributed Storage Engine for MySQL (Part 1)