Know Your Data(Frame) with Paguro
summary
Paguro claims to be a lightweight, Polars-native validation and metadata library that uses Polars-native predicates (expressions) as checks, supports single Data(Lazy)Frames and collections, and provides beautifully formatted terminal diagnostics plus SVG/HTML reports. It implements three validator types — ValidColumn/ValidFrame/ValidRelations — a DataSet context that composes (not inherits) data frames and can auto-validate on operations, supports deferred transforms and referential-integrity checks across tables, has one required dependency: polars, and the demo produces working formatted reports; talk cites $30M/yr spent on data quality.
AI-written summary · about
chapters (8)
- 0:00Paguro introduction
- 1:26Data validation importance
- 5:27Paguro validation API
- 7:17Python validation approaches
- 11:13Expressive independent validators
- 15:46Validation context and DataSet
- 19:41Live demo model approach
- 29:10Q&A
