PolEval 2025
summary
An overview of PolEval 2025, a Polish NLP evaluation contest aimed at improving tools and testing new algorithms in Polish language processing. It presents four shared tasks: detecting machine-generated text from Polish language models (ŚMIGIEL), building gender-inclusive large language models for Polish, document layout detection for Polish-language documents, and a Polish speech emotion recognition challenge; the event highlights participation from over 140 systems from academia and industry.
AI-written summary · about
chapters (10)
- 0:00PolEval intro
- 4:53Text detection task
- 13:50Task goal and structure
- 20:11Inclusive Polish models
- 27:39Second task description
- 32:22Document structure recognition
- 38:02Data format and evaluation
- 39:04Speech emotion recognition
- 42:58Data, metrics and baseline
- 45:47Summary and invite
related talks
29mHolistic Evaluation of Language Models (HELM)
01h 06mThe Future of Language Models: A Perspective on Evaluation
31mChallenges and Considerations in Language Model Evaluation
16mBest Practices for Open Multilingual LLM Evaluation
42mAdvancements in Evaluating Large Language Model Applications
20mFive hard earned lessons about Evals
30mResponsible AI with fmeval - an open source library to evaluate LLMs
43mEvaluation is all you need