tasks-you-gotta-know-how-to-run-em-you-gotta-know-how-to-safe-em/
summary
Covers concrete Celery practices: use Celery with RabbitMQ and Redis as brokers, keep results backend separate when needed, pass object IDs not model instances and use JSON serialization, make tasks short and idempotent, prefer atomic updates, use retries with exponential backoff or auto-retry, set soft and hard time limits and acks_late for safe reruns, integrate logging and monitoring (Sentry/Rollbar, Flower), and provides two shipped tools — a Celery task checklist and a General Celery beat status dashboard — both open source and production-ready.
AI-written summary · about
chapters (14)
- 0:00Intro
- 1:28Context
- 3:08Broker
- 8:04Handling errors
- 16:12Logging
- 16:43Monitoring errors
- 17:20ColonyFloor
- 17:37SolidBits
- 17:58Status
- 18:17Bugs
- 18:27Sivir
- 18:53PDB
- 19:16Recap
- 20:02Task Checklist
related talks
36mAn Introduction to Distributed Task Queue
28mOn a shoe-string and a t2.small: scaling on a [zero] budget.
28mHow to Do the Same Thing Over and Over Again and Yield Different Results
07m5 Steps to Resilient Job Queues
23mOrchestration Lightning: Rapid-Fire AWS Step Functions Patterns
51mCode the Future, Now
32mTips and tricks: data science prototype into production
52mSo You Want to Build An Event Driven System?