Data Morph: A Cautionary Tale of Summary Statistics
summary
Summary statistics like mean median std and Pearson r can match wildly different point clouds, so a working counterexample is packaged as Datamorph package installable from PyPI and conda‑forge; it loads CSVs into a dataset class with three bounds, builds targets via a shape factory and a shape class hierarchy, then morphs points using simulated annealing while preserving five summary stats. Typical runs take ~50k iterations for the dinosaur, produce animated morphs, and expose limits: bald spots, speed and convergence issues.
AI-written summary · about
chapters (12)
- 0:00Introduction and project overview
- 0:26Limitations of summary statistics
- 3:36Visualization as essential analysis
- 6:01Datasaurus dozen and memorable examples
- 7:25Introducing DataMorph package
- 9:08How DataMorph works step-by-step
- 13:22Shape class hierarchy and customization
- 14:45Morphing algorithm and simulated annealing
- 17:11Limitations and future improvements
- 20:21Turning research into a Python package
- 22:48Challenges and lessons learned
- 23:30Closing and resources
related talks
32m"Data Morph: A Cautionary Tale of Summary Statistics"
03h 07mData of Unusual Size: Interactive Visualization
01h 38mEffective Data Visualization
35mBad data - anecdotes and examples from the real world
25mVerifying and evaluating scientific results with the open source package "scores"
37mThe Python Ecosystem for Data Science: A Guided Tour
24mTalk Data to Me - the art of storytelling
08mIt might look normal but this distribution will ruin your stats