Closing Gaps From Commercial Tools in German Redisp...
summary
An internal tool named rep evolves from a one-off script into a data hub that ingests 15 GB/day and reconciles about 6 million measures from roughly 150,000 PV plants across ~20 systems, using a delta‑lake architecture with row versioning and a move from pandas to Polars (hashing 1M×144 columns drops from 5 minutes to ~20 seconds). It detects and sometimes auto-corrects inconsistencies, supports cost forecasting and risk assessment, and is working but not yet published as open source.
AI-written summary · about
chapters (6)
- 0:00Introduction and speakers
- 1:03Why redispatch matters
- 6:31Challenges in redispatch process
- 15:53Building Rep as open tool
- 24:31Technical architecture of Rep
- 28:55Q&A session
related talks
27mOpen-Source Congestion Mana...
35mOpen source tools for the power sector
34mData pipelines powered by Open source and fun
29mData Acquisition: Overcoming Challenges
29mBuild simple & scalable data pipelines with Polars & DeltaLake
47mHidden gems of WAL-G backup tool
37mData Swamp to Data Swimming Pool: Cleaning up the Mess
38mOpen Energy Modeling for Mission-critical Planning Purposes