A Journey Into Data Science: From an Idea to a Viable Product
summary
Builds an R package to standardize tax spreadsheets that ingests dozens of yearly Excel/CSV files, reconciles changing column names and department codes, turns ND/ND and D flags into documented missingness, and versions cleaned datasets in Git. Implements functions to compute Gini coefficient and Lorenz curve across years, finds only about 15% of communes have usable income values, and produces reproducible summaries and a small web app so the pipeline and inequality results actually run end to end.
AI-written summary · about
chapters (11)
- 0:00Introduction to journey
- 0:49Where data science starts
- 2:15Real-world data example
- 4:29Data quality challenges
- 7:10Storing and versioning data
- 8:50Data exploration and cleaning
- 11:53Handling missing values
- 13:25Visualizing inequality
- 15:27Calculating Gini coefficient
- 16:56Interpreting results
- 18:05Key takeaways and conclusion
related talks
20m15 Years of R in Quantitative Finance
01h 53mAnalyze your data in Python
32mEDA Techniques for Data Analysis
21mLearning and using the tidyverse for historical research
23mGrowth Hacking - Product Analytics at Scale using R and RStudio
48mCommon Data Science Mistakes
46mElegant R: Tips from a Software Developer
33mBuilding Data Science Tools for Sustainable Transformation