The Future Of Reproducible Research: Powered By Kubeflow
summary
Publishes a peer‑reviewed article plus a Kubeflow Pipeline that reproduces a low‑dose CT denoising workflow, running on Kubernetes with Apache Spark and Apache Mahout to perform a distributed stochastic SVD; uses DICOMs from coronacases.org (10 patients), shows denoising improves image clarity but over‑denoising erases signal, cites radiation tradeoff 6.1 mSv vs 1.4 mSv, and notes replication success jumps from 65% to 91% when the original author is involved.
AI-written summary · about
chapters (23)
- 0:00About the Narrator (me)
- 1:27About Electric Elk
- 2:30My Main Hustle This Month
- 5:38REPRODUCIBILITY
- 6:17The Replication Crisis: What Is It?
- 6:56The Replication Crisis: Causes
- 10:07The Replication Crisis: Consequences
- 16:32My Experience Trying to Reproduce Research
- 18:04Tower of Babel: Making Apache Spark, K8s, and Kubeflow Play Nice
- 19:20Rapid Testing Needed - Desperately
- 20:55CT Scans for Rapid Testing
- 21:39Low Dose CT Scans
- 22:22"De-Noising" Images
- 23:10Data Source
- 23:47Why (F)OSS Based Solution
- 25:16S3 Buckets
- 27:30Visualizing Results
- 27:56Positive Outcomes: Cool Example for Book
- 29:55Calls To Action: Assume Replication
- 31:29Calls To Action: Assume they won't be using your laptop.
- 32:23What is Kubeflow and Why Will it Help? Talk about Kubaflow pipelines a series of docker containers that execute stops then hand off data to next step
- 33:17Setting up Kubeflow
- 34:12Conclusion: Buy our book
related talks
35mOrganizing data engineering around Kubeflow
30mRealizing End to End Reproducible Machine Learning on Kubernetes
27mKubeflow for enabling AI Powered Drug Discovery and development...
22mGenerative AI Model Data Pre-Training on Kubernetes: A Use Case S...
33mCompliant Data Management and Machine Learning on Kubernetes
05mLightning Talk: Build a Reproducible ML Workflow with Kubeflow Pipelines
22mGenerative AI Model Data Pre-Training on Kubernetes: A Use Case Study
31mKubeflow - Data Science on Steroids