Email at Scale: How We Survived 800M Emails/Year (and DNS)
summary
Mirakl engineers explain how they rebuilt their email system to handle ~800 million messages/year by moving from batch processing to a centralized, Kafka-driven evented pipeline shared across products. They describe pipeline design and SMTP resilience patterns (duration-based processing, resilience patterns), the migration steps (provider change, white‑labeling, DNS record support, gradual cutover) and operational work (dashboards, alerting), plus practical issues: domain validation, DNS propagation, SPF lookup limits, IP warmup and reputation. Key lessons include a shared event-driven architecture, a slow-topic strategy for variability, resilience patterns, and a DNS/deliverability migration checklist.
AI-written summary · about
chapters (12)
- 0:00Mass email intro
- 7:27Batch architecture problems
- 10:15Migrating Kafka circuit breaker
- 13:29Reducing monitoring costs
- 17:23DNS prep for email switch
- 22:46White-labeling DNS strategy
- 25:34Top 5 DNS problems
- 31:10Dedicated IP warmup
- 35:04Suppression list risks
- 37:51Production monitoring tools
- 40:22Mass email key tips
- 43:11Summary & Q&A
related talks
28m10 Billion Messages in a Day, How We Built the Infrastructure That Delivers
32m"From On-Prem to the Cloud and Back Again"
44mUniSender Architecture: Growth from 100M to 1.5B Monthly Emails
31mLogging at Scale, Billion+ messages, 100K msg/sec
26mImproving Your Services the DevOps Way
31mScaling SRE at MailChimp: How We Did It
22mLessons Learned from Scaling RabbitMQ in a High-Growth (...)
55mSo You Want to Maintain a Reliable Event Driven System