Look Ma, No Background Jobs: A Peek into the Async Future
summary
Shows how to replace background job queues for IO-bound work by running requests inline on Falcon using the Async gem and ruby fibers. Describes an architecture that spawns supervised async tasks during a web request, waits or streams partial responses, and handles timeouts, retries, and resource limits. Demonstrates with external API calls and SMTP examples, compares behavior to Sidekiq-style queues, and explains tradeoffs: it scales with many concurrent connections but does not help CPU-bound work.
AI-written summary · about
chapters (5)
- 0:00Introduction
- 2:55Background jobs challenges
- 10:56Async future benefits
- 17:28Implementation details
- 21:57Conclusion and Q&A
related talks
23mConcurrency Showdown: Threads vs. Fibers
33mAsynchronous Rails
23mAsync Ruby
27mPrioritization justice: lessons from making background jobs fair at scale
30mAdvanced Multithreading in Ruby
30mBuilding Scalable & Resilient Background Jobs for FinTech
27mIn-Depth Ruby Concurrency: Navigating the Ruby concurrency landscape
35mConcurrency in Ruby: Threads, Fibers and Ractors Demystified