Methodology of Credit Risk Modeling Using Kedro Pipeline
summary
Argues that using the Kedro template plus Docker and MLflow logging turns ad‑hoc PD model work into a reproducible pipeline: a seven‑folder project (conf, data, logs, notebooks, src, tests, etc.) with pipelines built from nodes and a data catalog, run via kedro run (thread runner) and visualized with Kedro‑Viz. Notes tradeoffs: initial development slows (~100% overhead learning structure) and version compatibilities break across Kedro releases, but pilot use shows clearer validation, repeatability and faster iteration once standardized (dockerized reproducibility).
AI-written summary · about
chapters (20)
- 0:00Introduction
- 1:10Credit risk needs pipelines
- 2:55Probability of default models
- 4:47CRISP-DM methodology
- 6:38Unstructured model challenges
- 8:09Kedro vs ZenML vs Metaflow
- 10:41Why Kedro fits credit risk
- 12:54Choosing pipeline tools
- 13:14Kedro project structure
- 15:02Kedro data catalog
- 17:35Kedro pipelines and nodes
- 19:02Kedro visualization
- 20:52MLflow with Kedro
- 21:59Kedro in Docker
- 23:08New Kedro project
- 25:16Kedro functions and execution
- 27:22Running Kedro pipeline
- 28:48Pipeline visualization
- 30:13Kedro limitations
- 31:38Q&A
related talks
35mOrganizing data engineering around Kubeflow
35mSimplification of Data Pipelines by Containerization
34mData pipelines powered by Open source and fun
18mAutomate the boring ML stuff with pipelines
52mMachine Learning Data Pipelines
36mBuild prod ML systems with feature /training /inference pipelines
53mEnd-to-End ML
33mDiscover How to Create Your Own Metadata-Driven ML Platform from Scratch