Recursion Schemes Without the Barbed Wire (Part 6)
summary
Claims that recursion schemes let a schema-driven pipeline do validation and transformation in only half a traversal, implemented with algebraM + Traverse and a state monad registry for deduplication. Uses matryoshka to derive JSON→fixed JData converters, Geto validation to produce composable rules, and SCARA check generators for tests. Builds a working JSON ingestion MVP in about half an hour that produces precise localized error reports, generates schema-compatible test data, and short-circuits or reuses records as intended.
AI-written summary · about
chapters (7)
- 0:00Introduction
- 1:24Monadic traversals benefits
- 6:54Monadic algebra schema transform
- 8:03Data validation schemas JSON
- 15:29Generating validation rules
- 17:54Building data validated JSON
- 22:12Sanitizing validating schema-driven
related talks
29mSchemas: The Key To Data Happiness
36mData pipelines engineering made simple with Scala
30mContracts, Schema Evolution in Data Pipelines
16mBeyond ETL: Revolutionizing Data Engineering Workflows
24mLightning Talks
20mBuilding a Data Pipeline with Testing in Mind
20mSkrub DataOps, Probabl
12mSculpin: Direct-Manipulation Transformation of JSON