An introduction to Rejected, a Consumer Framework
summary
Rejected is an open-source Python RabbitMQ consumer framework running in production since 2009 that centralizes reconnects, graceful shutdown, per-consumer processes (master control program), and built-in telemetry to statsd/influx/Grafana. v3 supports Python 2.7/3.5; v4 moves to Python 3.7+ and asyncio. It auto (de)serializes by AMQP type, reports to Sentry, provides testing without Rabbit, defines three exception types where a consumer exception stops process and a processing exception requeues, and includes an avro contract-based messages add-on used to publish to Kinesis Firehose → Parquet in S3.
AI-written summary · about
chapters (22)
- 0:00Intro
- 0:42Why write a consumer framework
- 3:11What is Rejected
- 4:58Process Supervision
- 5:45Master Control Program
- 7:29Rejected Consumers
- 10:18Consumer Methods
- 11:49Message payloads
- 13:04Consumer exception types
- 14:28Consumer class attributes
- 15:39Error max retry
- 16:05Telemetry
- 17:58Consumer Example
- 19:08Consumer Test Cases
- 20:26AVRO Consumer
- 22:05AWE Consumer
- 23:51AWE Consumer Example
- 26:48Roadmap
- 28:55QA
- 30:44Message Exceptions
- 32:38Logging Messages
- 35:13Publishing frameworks
related talks
01h 31mIt's the message, stupid: python & amqp
27mReal-time Event Streaming with Python
38mEvent-driven applications: Apache Kafka and Python
36mKafka Streaming, the Pythonic Way
26mEvent-driven applications: Apache Kafka and Python
41mFixin the Hard Bits of Event Processing with Restate & Kafka
45mEvent-Driven applications: Apache Kafka and Python
24mEfficient Communication Between Python Microservices using LavinMQ