DiffStream: Differential Output Testing for Stream Processing Programs
summary
DiffStream is a Flink testing library that compares a sequential and a parallel run by a matching algorithm that decides stream equivalence up to an explicit relation on event pairs (same-key ordering, etc.). It provides an online, space-optimal matching algorithm with a proved optimality theorem, handles mixed ordered and out-of-order outputs, and costs around 5% maximum throughput. Case studies include taxi GPS aggregation, parallel topic counting, and adapted MapReduce benchmarks where it finds real bugs and avoids false positives that prior tools (≈58% false positives) produce. Lightweight ordering relation spec
AI-written summary · about
chapters (13)
- 0:00Start
- 0:41Evolution of Stream Processing Systems
- 2:07DiffStream: Differential Testing for Flink
- 2:56Programming Errors: Example
- 6:10Programming Support for DSPS: Challenges
- 6:57Our Solution: Differential Testing
- 8:26Lightweight Specification Language Typical scenario: ordered by key, unordered across keys
- 9:59Technical Challenge: Checking Equivalence
- 11:08Checking Equivalence: Theorems
- 11:51Experiments: Overview
- 13:00Case Study: MapReduce programs
- 14:36Future Opportunities
- 15:33Summary
related talks
38mDIFFY Catching Bugs without Writing Tests
34mTesting Production Streaming Applications
01h 11mKafka streams testing: A deep dive
30mA Debuggers Guide to Apache Flink Streaming Applications
36mFlinkspector: Taming the squirrel
34mCan your diff(1) do this?! Improving software review and QA with diffoscope
23mDDataflow: An open-source end to end testing from machine learning pipelines
17mDifferential Network Analysis