Making Data Pipelines in R: A Story From A “Self-Taught” Perspective
summary
Built a working R pipeline for messy opioid/Narcan datasets that initially coerced strings like "4 plus" into numbers, forcing a rework of process and checks. Implements organized file structures, modular scripts that a single docking script sources, automated HTML report rendering, and data validation using packages like pointblank and here. Includes metadata, data workflow maps and SQL‑style schemas in a public GitHub repo so the pipeline runs reproducibly and is easier to hand off.
AI-written summary · about
chapters (7)
- 0:00Data pipelines in R
- 1:51First encounters
- 6:30Environmental structures
- 10:52Internal environments
- 14:10Data validations
- 16:43Pipeline sustainability
- 19:13Recap and key takeaways
related talks
20mMaking Data Pipelines in R: A Story From A “Self-Taught” Perspective
34mData pipelines powered by Open source and fun
16mAutomating the Dutch National Flu Surveillance for Pandemic Preparedness
30mDesigning Resilient Data Pipelines
43mData Pipelines à la Mode
41mAccelerating privacy-enhancing data processing
26mSimplifying Data Analytics Using RudderStack Open Source Data Pipelines
22mPlowing Through Data Building Flexible Pipelines with Go