Establishing a Serverless Dynamic Tile Generation Workflow
summary
Converted a 197 TB JPEG2000 archive on S3 into manageable tiles by slicing source images into 20 MB chunks with a GDAL Python pipeline, indexing chunk footprints in PostgreSQL, and using an EC2 auto‑scaling queue to preprocess in days instead of months. Runtime tile composition uses Rasterio/GDAL to merge chunks into JPEG tiles, served via Lambda + API Gateway with caching, pregen to zoom 11 and on‑demand generation, achieving typical latencies of <2 seconds.
AI-written summary · about
chapters (4)
related talks
23mFrom Satellites to Slobs
50mDeveloping Highly Scalable Image Storage Solution with AWS Serverless
46mHighly Scalable Image Storage with AWS Serverless
21mThe best compute for your storage - Amazon S3 & AWS Lambda
33mBuilding Open-Source Cloud-Native Apps in JavaScript
05mAlex Kudriashova Interview
41mUnlocking Faster & Efficient Data Processing w/ Serverless
32mServerless Data Processing and Machine Learning