"I ran the code for you and I think the problem is X" ...
summary
A large language model outputs I ran the code for you while no execution happens because there is no compiler or VM generated, only token prediction. The model is trained on internet text where debugging replies often begin with that phrase so it hallucinates execution. Result: a false claim that a run occurred, no artifact was built, and experienced engineers confuse language generation with language understanding, leading to misplaced confidence in debugging outputs.
AI-written summary · about
related talks
40mIs my Large Language Model a Strange Loop?
43mDiscussion
58mIt AI-n't What You Think!
57mArtificial Languages are Dead. Long Live Artificial Languages!
29mLies, damned lies and large language models
41mTeaching Exotic Programming Languages to Large Language Models
01h 22mBeyond Decoding: Meta-Generation Algorithms for Large Language Models
32mCan We Trust AI-Generated Code?