A pragmatic and simple approach to fixing a memory leak
summary
A Rails background worker on Heroku grows past 1GB despite scaling to 2.5GB and splitting queues; concurrency 25 lowered to 15 still shows slow growth. Investigation counts ~120 default jobs, then uses a quarantine queue and binary-split batches (seven to eight candidates) to isolate one leaking job. Memory reports via memory_profiler.report point to Segment client allocations. Fix is to instantiate the analytics client once in a Segment initializer instead of per job and the worker memory stabilizes.
AI-written summary · about
chapters (13)
- 0:00Introduction
- 3:30Memory leak ticket investigation
- 7:23Testing config changes
- 9:31Worth the effort?
- 10:35Researching memory solutions
- 13:46Instrumenting suspect jobs
- 15:53Binary search approach
- 18:23Why binary search works
- 19:48Quarantine queue middleware
- 21:56Leaking job identified
- 23:01Simple fix resolved leak
- 23:42Post-fix root cause
- 27:12Key takeaways
related talks
46mHalve Your Memory Usage With These 12 Weird Tricks
42mMemory Fragmentation and Bloat in Ruby
41mDetecting and Fixing Unbound Concurrency Problems
28mFair by design: orchestrating background jobs in Ruby
38mYour App Server Config is Wrong
50mFrom Rails Rumble to 50,000,000 results
45mCatching the 137-Killer: A Java Memory Forensics Investigation
24mScaling Argo CD: From Symptoms To Solution