Training AI To Code Using the Largest Code Dataset
summary
Project CodeNet is a 14 million–sample, 500 million–line dataset across 55 languages (CDLA v2) used with the Machine Learning Exchange to preprocess, subset via DataShim into Kubernetes Custom Resources, run Jupyter notebooks and Kubeflow Pipelines on Tekton, train models and serve them as KServe CRs. Claim: a migration workflow reduces a Java modernization effort from ~1 year to ~4 weeks, producing 25 microservices and 450 updated classes. Models and notebooks use Apache 2.0 weights and show preliminary language-model and classifier results.
AI-written summary · about
chapters (18)
- 0:00Training AI to code
- 1:06AFO Code initiative goals
- 2:30Why large datasets matter
- 4:16CoNet dataset intro
- 7:25CoNet dataset structure
- 10:38CoNet metadata and tools
- 13:28CoNet use cases
- 15:54Project Wisdom and code gen
- 17:20ML Exchange overview
- 20:10ML Exchange demo
- 24:27Running models on cluster
- 26:56Summary and future
- 28:23Q&A session part one
- 30:12Q&A session part two
- 31:28Q&A session part three
- 33:31Q&A session part four
- 35:24Q&A session part five
- 36:46Closing remarks
related talks
42mTraining AI To Code Using The Largest Code Dataset (Project CodeNet)
39mTraining AI To Code Using The Largest Code Dataset (Project CodeNet)- 39mTraining AI To Code Using The Largest Code Dataset (Project CodeNet)
37mMachine Learning on Source Code
28mDelivering AI on Code: Live Demo of source{d}
46mCode + AI: Will Robots Take Our Coding Jobs? ML Applied to Programming
43mMachine Learning for Developer Productivity
34mGenerative AI Model Data Pre-Training on Kubernetes: A Use Case Study