GPT-3-Powered Type Error Debugging: Investigating the Use of Large Language Models...
summary
Uses GPT-3 to generate OCaml repairs by building rich prompts from source-code analysis (candidate locations via a magic function, inlining, and type unification), then applies three repair strategies—Phill, Choose, Instruct—plus an offline try-every-suggestion mode. Validates patches automatically with QuickCheck (10,000 tests) against a public dataset: 1,318 bugs yield 516 fully fixed (39.2%) and 441 partial fixes (33.5%). On 591 common benchmarks Mentat fixes 222 vs 198 and 46 for two prior tools.
AI-written summary · about
chapters (13)
- 0:00GPT-3 for type error debugging
- 1:03Type systems and errors
- 4:39Motivation for automated repair
- 6:03Building GPT-3 prompts
- 8:12Type error location with magic functions
- 9:37Inlining and type unification
- 11:04Repair strategies with GPT-3
- 15:00Offline repair strategy
- 16:03Large-scale evaluation results
- 20:17Comparison with tools
- 21:21Requirements and language-agnostic
- 22:24Future work and models
- 23:12Q&A
related talks
29mOne-Click Code Fix: Securing Code Using AI
57mKeynote: Using LLMs and generative AI to fix software vulnerabilities
50mIdentifying and fixing Issues in Code using AI based tools
28mEnhancing Kernel Bug Discovery with Large Language Models
56mWhy Auto-Fixing Vulnerable Code Needs More Than Good Intentions
36mCode Mending in the AI age
18mChallenges in C++ to Rust Translation with Large Language Models: A Preliminary(…)
01h 02mAI Effects on Research and Education: A Programming and Software Language Perspective