How to create a fast and cost effective serverless geospatial analysis pipeline...

on YouTube ↗19m

" summary

Shows a working, cost‑effective geospatial pipeline built on AWS Lambda that ran on the free tier for over a year and handled daily/weekly jobs. Uses the serverless framework, Node or Python virtualenvs, and a Docker build with GDAL to bundle binaries; deploys packages via zipped artifact to avoid Lambda limits. Notes ephemeral storage so results go to S3, Lambda max execution 300s requiring task splitting, package size limits (50 MB zip, ~256 MB unzipped), and that intersection geocoding still relied on Google Maps.

AI-written summary · about

" chapters (5)

1. 0:00Start
2. 0:03Fast and cost effective geospatial analysis pipeline with AWS lambda
3. 14:13Adapted from mapbox/landsat-tiler
4. 16:21Divide & Conquer
5. 17:46lambci/docker-lambda

" related talks

• 34mDesigning Data Intensive Applications In Serverless ArchitectureNikolay Matvienko · Heapcon 2019
• 46mServerless for Data ScientistsMike Lee Williams · PyData Los Angeles 2018
• 34mBuilding scalable end-to-end deep learning pipelines in the cloudRustem Feyzkhanov · Global Artificial Intelligence Virtual Conference 2022
• 32mServerless Data Processing and Machine LearningSunil Mallya · LISA18
• 01h 00mServerless Architecture - Tales from a world without serversRobin Weston · NDC London 2017
• 01h 00mServerless - reality or BS - notes from the trenchesLynn Langit · NDC Oslo 2017
• 49mServerless Architectural Patterns and Best PracticesSascha Möllering · GOTO Berlin 2018
• 54mServerless Architectures: no servers allowed!Antons Kranga · JavaDay Lviv 2016