Memory Management in Python: Foundations, Problems, and Modern Techniques
summary
Explains CPython’s two-phase memory reclamation: reference counting plus generational GC with gen0 triggered after ~700 allocations and gen1 after ten gen0 collections, and PyMalloc versus OS malloc for small versus large objects. Shows concrete pitfalls — circular refs, mutable aliasing, unbounded containers, fetchall, logging.exc_info, and nondeterministic __del__ (use context managers instead) — and practical fixes like __slots__, generators, object reuse, capped deques or expiries. Demonstrates diagnostics with tracemalloc A/B snapshots and pympler summarize, finding 1.2 GB from 5,000 bytearrays in a WebSocket sessions leak.
AI-written summary · about
chapters (23)
- 0:00Introduction and motivation
- 1:28Talk agenda overview
- 2:09Memory layout in programs
- 3:13Names objects and references
- 4:39Memory allocation basics
- 5:21Reference counting in Python
- 6:44Reference cycles and garbage collection
- 8:53Generational garbage collection
- 10:59Garbage collection thresholds
- 11:44Memory allocation demo
- 12:51Python vs OS memory allocators
- 15:23Common memory pitfalls
- 16:06Mutable aliasing issue
- 16:48Unbounded containers problem
- 18:20Batch processing for large data
- 18:44Exception logging memory trap
- 20:08Finalizer method limitations
- 21:55Debugging memory issues
- 22:16WebSocket memory leak case
- 23:22Using TraceMalloc tool
- 24:51Analyzing TraceMalloc output
- 26:40Pympler for object analysis
- 28:35Summary and Q&A
