CoGrader: Transforming Instructors' Assessment of Project Reports through Collaborative LLM Integ...
summary
CoGrader pairs teachers with LLMs to reallocate tasks: instructors human-LLM collaborative metrics design, pick benchmarking with teacher-selected exemplar reports and authorize final feedback while the model runs automated consistency checks and regrades all submissions when benchmarks change. Built as a workflow with a magic view, benchmarking view, and feedback view, it uses LLMs for standardized checks and comment drafts, preserves human control for creativity and novelty, and evaluates against IRB-approved, manually graded class reports showing faster, more consistent grading.
AI-written summary · about
chapters (8)
- 0:00Introduction and presenter
- 0:29Project-based learning challenges
- 1:55Grading project reports difficulties
- 4:06LLM integration in education
- 6:37CoGrader system design
- 8:01Evaluation and findings
- 9:27Responsible AI grading
- 10:11Q&A session
related talks
25mEngineering Better Evals: Scalable LLM Evaluation Pipelines That Work
32mA Practical Guide to LLM Evaluation
18mLeveraging LLMs for student feedback in introductory data science courses
29mGenerative-AI: Usecase-Specific Evaluation of LLM-powered Applications
44mPractical LLM Evals: Down-to-Earth Tooling for Everyone
38mMaking LLMs reliable - A practical framework
35mEvaluating LLM systems: the nuts and bolts
50mBeyond the Prompt: Evaluating, Testing, and Securing LLM Applications