Graceful Deletes: Queues, Tasks & Distributed State Management
summary
Cohere’s North team builds an asynchronous user-deletion pipeline that immediately revokes RBAC/auth tokens and then cron-dispatches per-user Inngest over Celery subtasks to clean vector DBs, blob storage, sync service metadata and finally run cascading DB deletes. File removal uses two parallel subtasks (raw bytes and vector entries) plus token invalidation to stop syncs. Durable orchestration uses a Postgres-backed task state because Inngest’s composite-key TTL is 24h. The MVP is deployed: subtasks report completion, retries and idempotency are handled, and final delete runs once all services confirm.
AI-written summary · about
chapters (12)
- 0:00Introduction
- 0:27Presenters overview
- 2:00Deletion challenges
- 5:54Celery vs Inngest
- 9:07Async task flow
- 13:21Immediate vs eventual deletion
- 15:37Incremental delivery
- 16:04Design principles
- 18:52Cron execution details
- 20:27Concurrency layers
- 22:52MVP wrap-up
- 23:32Q&A
related talks
17mScheduled Deletions At Scale
18mImproving app performance by using background jobs
37mUsing Elixir for scheduling cloudnative DAG workflows
42mCelery on AWS ECS - art of background tasks & continuous deployment
32mFrom crontab to celery with no regrets
27mEmpowering Resilient Scalability: The Power of Choreography in Distributed Systems
38mHow we Migrated (...) Without Downtime using Elixir
15mEvolving Cluster Management