Lightning Talk: Highlights of the Snowflake Elastic Data Warehouse
summary
Snowflake claims a multi-cluster shared-data design that separates compute from storage and keeps metadata in FoundationDB. Data are stored in automatic micro partitions and a Spark connector injects SQL into Spark plans. Demo moves 3.2 million JSON tweets from S3, where a 2-node Spark job takes ~30 seconds, to a 1-node Snowflake warehouse returning the same group‑by counts in ~2–3.5 seconds. Materialized views and metadata scale are listed as ongoing builds.
AI-written summary · about
chapters (5)
- 0:00Start
- 0:57create independent compute resources that are powering your team
- 1:49scale these compute resources independently of each other
- 4:02cutting it into micro partitions
- 9:55running an actual snowflake query to populate another data frame
related talks
23mAcceleratinag Data Science and Machine Learning with Snowflake DataTech
20mBuilding An Elastic Query Engine on Disaggregated Storage
28mBuilding A Governed Data Lake
20mSnowflake concepts & architecture
24mA Single Data Platform for All of Your Workloads
33mData Engineering simplified with Snowpark for Python on Snowflake
21mCaching Snowflake data with Aerospike for advanced analytics
53mKeynote: From RAP to Snowflake - A Look at 50 Years of SQL DB Scalability