Polyglot Data Science: Why and How to Combine R and Python

on YouTube ↗17m

" summary

Combining R and Python solves concrete problems like using an algorithm only available in an R package or embedding R analysis in a Python ETL pipeline; practical options shown include rpy2 and reticulate for tight integration, Arrow for zero‑copy sharing, Quarto and Positron for workflows, Unix pipes or a save‑and‑load pattern for loose coupling. Cautionary numbers: one pipeline consumed 700 GB RAM and took two days twice a year until a full Polars rewrite to Python reduced cost and made it production‑feasible.

AI-written summary · about

" chapters (12)

1. 0:00Introduction
2. 0:24Why combine R and Python
3. 1:52Mindset for polyglot data science
4. 2:56R and Python as tools
5. 3:39rpy2 for visualization
6. 5:05Annoy for saving and loading
7. 7:15UNIX pipes foundation
8. 8:40Performance issues caution
9. 10:35Polars for efficiency
10. 11:59reticulate and Arrow in R
11. 12:45When to embrace polyglot
12. 13:49Q&A

" related talks

• 37mRewriting your R analysis code in PythonHelena Schmidt · PyCon DE & PyData Berlin 2022
• 35mEffective Python and R collaborationDaniel Rodriguez · PyData New York City 2019
• 32mInterfacing R and PythonAndrew Collier · PyData Berlin 2018
• 37mData Science with Python and RLuciana Tanus · DevDay 2018
• 45mPython useRsDaniel Chen · PyData DC 2016
• 18mYes, you can use Python with RStudio Team!Fei Xu, Gagandeep Singh · rstudio::conf 2019
• 19mCombining R and Python for the Belgian Justice DepartmentThomas Michem · posit::conf(2023)
• 22mUsing Python & R in HarmonyMatthew Brower, Krista Readout · PyOhio 2019