Backstage interview

on YouTube ↗06m

" summary

A project builds a large dataset for code‑generation LLMs on top of Software Heritage, ingesting all versions of open source repositories and running on an HPC cluster to handle massive scale. It adds issues and pull request descriptions, CI results and file‑level metadata, performs precise license detection and Python‑version analysis, and records attribution so models can cite source files and rights rather than training on code with unknown provenance.

AI-written summary · about

" related talks

• 30mWhat Can LLMs Do with Messy Data?Cedric Clyburn, Andrew Igdal · PyData London 2026
• 01h 03mWhat can we learn from 750 billion GitHub events and 42 TB of codeFelipe Hoffa · NDC Sydney 2017
• 40mDiscovering Emerging Open Source Communities Through...Hema Veeradhi, Oindrilla Chatterjee · OSPOCon @ Open Source Summit North America 2023
• 40mFrom Science to Open Source and Back AgainThomas Caswell · PyData NYC 2022
• 01h 04mWhat can we learn from 750 billion GitHub events and 42 TB of codeFelipe Hoffa · Voxxed Days Singapore 2017
• 19mOSPO Data PyramidCali Dolfi · Open Source Summit Europe 2025
• 01h 16mReproducible machine learning and science with pythonPrabhant Singh · PyCon DE & PyData Berlin 2022
• 45mGitHub Archive - Processing and analyzing the billions of events generated by the wJohannes Moser · Voxxed Days Vienna 2016