Recursion Schemes Without the Barbed Wire (Part 1)
summary
Claims that recursion schemes let a runtime‑configured data ingestion pipeline handle semi‑structured sources and produce Parquet and Avro are demonstrated with concrete code: Scala + SBT on JVM 8, a Git repo with exercises and failing tests, and the matryoshka library. The workshop implements schema‑driven validation and transformation components, shows uses of cata/ana/hylo and fixed points, and stresses practicality by targeting 10,000 data structures, using matryoshka, and outputting to Parquet and Avro within the exercise harness.
AI-written summary · about
chapters (7)
- 0:00Workshop setup
- 3:57Recursion schemes intro
- 11:54Folds, unfolds, refolds
- 13:35Pattern functor and algebras
- 22:15How the hylomorphism function works
- 35:46Hands-on exercises
- 43:18Q&A
related talks
41mThese 10000 classes I never wrote
36mData pipelines engineering made simple with Scala
57mBig Data Pipelines With Scala - Part 2 of 2
38mStreaming with Structure
52mData Over Code: A Short Hyperbole
01h 10mReal World Iterative Database Development
30mContracts, Schema Evolution in Data Pipelines
39mHow to avoid columnar calamities: what no one told you about Apache Parquet