How Apache Sedona is Revolutionizing Geospatial Data Analysis
summary
Apache Sedona extends cluster engines to run distributed spatial SQL and datasets and claims real-world speedups: it extends Apache Spark and Apache Flink, adds spatial partitioning and distributed indexes, and supports geoparquet reader/writer (beta) so queries prune S3 reads. Built components include a compact geometry serializer that is about 3× smaller and 20× faster to serialize (5× faster to deserialize). Benchmarks show a New York taxi 1.3B row range/join finishes in under ten minutes while raw Spark fails within a day, and an 80B OSM job runs in about five hours.
AI-written summary · about
chapters (13)
- 0:00Introduction
- 3:17Geospatial data challenges
- 7:52Apache Sedona ecosystem
- 12:04Spatial SQL queries
- 26:28Spatial query processing
- 27:58Geospatial data partitioning
- 31:07Distributed spatial indexing
- 31:49GeoParquet benefits
- 35:23Efficient geospatial serialization
- 37:29Performance benchmarks
- 40:19Real-world applications
- 42:45Integration with tools
- 43:09Q&A
related talks
41mProcessing Geospatial Data at Scale with Python and Apache Sedona (incubating).
04mAccelerating geospatial computing using Apache Arrow
41mOptimising Apache Spark and SQL for Improved Performance
47mHow we build InsightEdge: powerful Spark distribution
38mOn the path to data Nirvana: Supporting OLTP and OLAP on Hadoop
30mBuilding geospatial apps on Postgres, PostGIS, & Citus at large scale
39mAccelerate big data analytics with Apache Kylin
26mWho Said Wrangling Geospatial Data at Scale was Easy?