y0news
AnalyticsDigestsSourcesTopicsRSSAICrypto

#rubrics News & Analysis

2 articles tagged with #rubrics. AI-curated summaries with sentiment analysis and key takeaways from 50+ sources.

2 articles
AINeutralarXiv – CS AI · Jun 16/10
🧠

PReMISE: Policy Rubrics as Measurement Specifications for LLM Judges

Researchers introduce PReMISE, a framework for auditing and improving rubrics used by LLM judges to evaluate open-ended responses. The work reveals that existing rubrics—whether raw or human-created—fail to simultaneously achieve reliability, preference alignment, and adversarial robustness, with implications for how AI systems measure quality at scale.

AINeutralarXiv – CS AI · May 96/10
🧠

Counterargument for Critical Thinking as Judged by AI and Humans

A university study of 35 students examined whether writing counterarguments to AI-generated content develops critical thinking skills. Researchers found that student-written counterarguments demonstrated logical reasoning and that six frontier large language models could reliably assess student work using established rubrics, achieving moderate inter-rater reliability (0.33 Gwets AC2) comparable to human assessments.