Flinkspector: Taming the squirrel

on YouTube ↗36m

" summary

Flinkspector is a small testing framework for Apache Flink that builds reproducible event traces, synthesizes watermarks and timestamps, and runs jobs in a local cluster so non‑terminating and parallelized pipelines with windowing become testable. It provides an event-time input builder, incremental verifiers with triggers to stop tests, and Hamcrest-based matchers for fine-grained assertions, integrates with JUnit, ships on Maven, and aims to enable fast integration tests for CI/CD so streaming logic bugs surface during development rather than production.

AI-written summary · about

" chapters (8)

1. 0:00Introduction and context
2. 2:16Challenges in testing Flink
3. 5:55Initial testing approaches
4. 9:40Introduction to Flink Spector
5. 16:20Framework design and syntax
6. 24:28Advanced assertions and execution
7. 29:39Practical demonstration
8. 32:12Conclusion and Q&A

" related talks

• 22mDrSquirrel - A self-service diagnosis tool for Flink jobsFanshu Jiang, Lu Niu · Flink Forward Global 2021
• 34mTesting Production Streaming ApplicationsGyula Fora, Matyas Orhidi · Flink Forward Virtual 2020
• 30mA Debuggers Guide to Apache Flink Streaming ApplicationsAlexander Fedorov · Flink Forward Global 2021
• 16mFlink acceptance testing and state compatibility checkingCatlyn Kong · Flink Forward Virtual 2020
• 40mInteractive Session - Live Code Contribution to Apache FlinkSeth Wiesman · Flink Forward Global Virtual 2020
• 38mInteractive Session: Flink on Autopilot with SplunkMaximilian Michels · Flink Forward Global 2021
• 35mTouK Nussknacker – creating Flink jobs with GUIMaciek Próchniak · Flink Forward Berlin 2017
• 41m0Flake - Reaching reliable non-flaky testsItai Freindinger · DevOpsDays Tel Aviv 2018