Tiling & Parallel Processing of Large Images
summary
Pipeline replaces whole-image processing with tile-level work using rasterio to read/write offset tiles, runs models per tile and averages class probabilities for voting, and applies a 50-pixel buffer on 512×512 tiles to smooth segmentation seams. Statistical ops keep min/max/count and merge means exactly, approximate standard deviation with tile stats yields ~2% std error in practice, and cloud parallelism cuts one workload from 12 hours to 25 minutes.
AI-written summary · about
chapters (8)
- 0:00Tiling and parallel processing
- 2:00Large imagery challenges
- 3:50Classification with tiling and voting
- 5:58Statistical tile calculations
- 9:55Quantile estimation and segmentation
- 11:43Buffering segmentation maps
- 14:53Tile-level processing trade-offs
- 16:41Q&A on agriculture
related talks
49mGraph Based Processing of Big Images
54mHow to handle the luxury of having too much training data
17mEfficient & Frugal Image Data Processing Pipelines
01h 36mFrom Maps to Models: An End-to-End Journey in Geospatial Machine Learning with Python
31mFrom Radar Echoes To Real-Time Recognition: Deep Learning on Another Planet -An Open Source Workflow
28mProcessing medical images at scale on the cloud
34mDeep Learning at Scale
01h 04mBuilding Efficient and Frugal Image Data Processing Pipelines