Take The Path Of Least Resistance For Your Test Data
summary
Shows how tests create records directly in Postgres then trigger RabbitMQ-driven pipelines with a maintenance tasks helper to populate Elasticsearch and Amplify, using smart loops that poll service APIs instead of sleeps. Includes a consumer health check suite that fails fast if queues lack consumers or exceed a 50,000 message threshold. Demonstrates private test APIs, queue admin checks and clear errors that expose ETL or consumer problems early, and reports this approach working in staging.
AI-written summary · about
chapters (6)
- 0:0002:30 - Introduction
- 5:3607:35 - What is Review Trackers?
- 8:0211:35 - How data moves around in production??
- 13:4721:41 - How to utilize this system to move around data tests
- 18:0233:45 - Other great uses of RabbitMQ for tests
- 23:0738:02 - QnA
related talks
39mAll of those tests you have and none of them cover Message Queues
39mWhat Can Go Wrong With Testing an ETL Pipeline?
20mBuilding a Data Pipeline with Testing in Mind
36mAutomated Performance Testing of Postgres
37mRethinking How We Test Our Async Architecture
25mTesting Data Pipelines
51mBuilding a Self-Service Marketplace for Test Data
30mHow to Test BigData/ETL Solutions?