Ondrej Babec, Jiri Novotny: Performance Testing and Why Even the Imperfect One is Important
summary
Talk documents a two‑year effort to performance test a Debezium Kafka connector on a messy mix of four bare‑metal lab machines plus a crusty office server and unreliable OpenStack, using PostgreSQL, Grafana, Prometheus, Netdata + netdata‑relay and a homegrown Database Manipulation Tool (DMT) to generate load. Timestamped events across the pipeline reveal the bottleneck: the connector cannot read DB changes fast enough; fixing data structures and buffering raises throughput. Key gotchas: Prometheus <1s scraping limitation and the need to centralize and tune metrics.
AI-written summary · about
chapters (10)
- 0:00Performance testing challenges
- 1:25Two perspectives on testing
- 8:52Imperfect testing process
- 19:32Two-year journey with testing
- 23:34Building test environment
- 27:49Visualizing metrics in Grafana
- 30:45Lessons from scraping intervals
- 33:20Fixing bottlenecks
- 38:37Key takeaways and tips
- 40:28Q&A
related talks
40mPerformance Testing and Big Data
21mCrunching Millions of Requests for Breakfast: A Tale of Performance Testing Prowess
01h 00mPerformance In Modern Times: An Evolutionary Journey
20mBuilding a Data Pipeline with Testing in Mind
30mAbove the Clouds: Investigating Systems from 32,000 ft.
36mAutomated Performance Testing of Postgres
44mPerformance Testing with a "Big Data Tool"
41mShift Left your Performance Testing