A Journey Through a Geospatial Data Pipeline: From Raw Coordinates to Actionable Insights
summary
Pipeline processes a 3‑day Swiss rainfall raster set: download COGs, stack into an Xarray cube, clip to the Swiss boundary, run per-tile filters and a blob/threshold detector, aggregate blobs, then intersect blobs with OpenStreetMap building footprints to produce GeoJSON and vector tiles for visualization. Prototype runs as a single-node Python script using Xarray + Dask and Parquet/duckdb for vectors; scaling relies on COGs and Zarr, spatial indexing (R-tree/H3) and PostGIS/Sedona to make it production-ready.
AI-written summary · about
chapters (12)
- 0:00Geospatial data pipelines intro
- 1:08Swiss rain data processing
- 3:56Geospatial data tools overview
- 8:11Pipeline design step-by-step
- 10:21Scaling with cloud-native formats
- 12:53Processing tools for geospatial
- 15:24Spatial indexing importance
- 17:12Visualizing geospatial data
- 18:36Geospatial stack overview
- 20:02Production-ready pipelines
- 21:51Future of geospatial data
- 22:11Q&A: Vector apps & COG limits
related talks
26mWho Said Wrangling Geospatial Data at Scale was Easy?
35mVisualizing research data: Challenges of combining different datasources
01mA Practical Introduction to Geospatial Python
31mUnpacking the geospatial engineering toolbox – an overview of data science techniques for spatial da
26mCartography with Python
31mOpenDataCube - Petabyte Scale Satellite Imagery Processing in Python
40mData-centric exploration using intake, dask, hvplot, and more
40mStreaming Geospatial Vector Data with Apache Projects