From messy to meaningful data: LLM-powered classification in R
summary
Demonstrates using ellmer and the parallel chat structured API in R to turn text, PDFs and images into labeled data, measuring real outcomes: LLM classifications hit 62% accuracy on a 24-way disease dataset versus 93% from a tidymodels baseline, cost about $0.95, and confidence scores broadly correlate with correctness. Image experiments show binary iris vs rose works but three-way iris fails. Crime-labeling agrees ~81% with an existing ML tool. Uses GPT-5 and validation by human/ML disagreement.
AI-written summary · about
chapters (13)
- 0:00LLM data classification intro
- 1:28Traditional ML for data extraction
- 2:32Structuring LLM responses in Elmer
- 3:58Testing LLM limits with Iris
- 4:40Prompting strategies for LLMs
- 5:44Disease diagnosis with LLMs
- 8:22Crime classification without truth
- 10:53LLM vs ML model disagreements
- 12:43LLMs with traditional ML workflows
- 13:05Frontier model limitations
- 14:31LLM unpredictability and validation
- 15:13Key takeaways
- 16:18Q&A
related talks
22mIntegrating LLMs with Traditional Data Analysis
33mLessons Learned Applying Large Language Models in Healthcare
24mDemocratizing data access through LLM
56mDehumanizing Agents: Why Explainability is Crucial in the LLM Era
30mWhere there's a will, there's a way: The mad genius of LLMs as a classifiers
31mLeveraging LLMs to build supervised datasets suitable for smaller models
29mAutomating Privacy Reviews with LLMs: Challenges & Opportunities
25mUsing Large Language Models to Improve Data Loss Prevention in Organizations