Fast Image Pulls Using IPFS And Opportunistic Caching
summary
Cluster runs a node-local proxy called registry facade that dynamically assembles OCI manifests, pushes missing blobs into IPFS (via nerdctl-registry and eStargz experiments), and uses an opportunistic pull-through caching model with an IPFS cluster made region-aware. P95 startup drops from >10 minutes to ~120s and P50 from ~24s to ~10s; ~90% of pulls now come from IPFS (≈2.3 TB/day) versus ≈100 GB/day to upstream registries. This runs in production for months and reduces bandwidth and latency without forcing image changes.
AI-written summary · about
chapters (14)
- 0:00Introduction
- 1:22Docker image challenges
- 3:07Caching strategies
- 5:59Community efforts
- 7:24Registry facade
- 9:33What is IPFS
- 10:15First nerdctl attempt
- 12:04Stargaze snapshotter
- 14:12Replacing Stargaze
- 15:15IPFS scaling issues
- 16:19Region-aware setup
- 17:26Production results
- 19:34Conclusion and key takeaways
- 21:00Q&A
related talks
30mFaster Container Image Distribution on a Variety of Tools with Lazy Pulling
37mCache Me If You Can: P2P Image Sharing in Kubernetes with Spegel
29mPull Once, Scale Everywhere: Fixing Image Pull Bottlenecks With Dragonfly
25mStreamlined Efficiency: Unshackling Kubernetes Image Volumes for Rapid A...
29mImage Snapshotters for Efficient Container Execution in Particle Physics
04mLightning Talk: Solving Multi-Region K8s Challenges: Improving Pod Startu...
30mSpeeding Up Analysis Pipelines with Remote Container Images
25mStreamlined Efficiency: Unshackling Kubernetes Image Volumes for Rapid AI Model...