#safety-judges News & Analysis

2 articles tagged with #safety-judges. AI-curated summaries with sentiment analysis and key takeaways from 50+ sources.

2 articles

AIBullisharXiv – CS AI · Jun 97/10

🧠

Reliable to Expressive: A Curriculum for Rubric-Following Safety Judges

Researchers developed a curriculum-based training method for safety judges that dramatically improves their consistency across different evaluation rubrics. The approach combines dynamic rubric generation with a staged learning process, achieving 94.12-94.88% accuracy with minimal variance across three different rubric styles, outperforming larger general-purpose and specialized LLMs.

AINeutralarXiv – CS AI · Jun 256/10

🧠

Do Encoders Suffice? A Systematic Comparison of Encoder and Decoder Safety Judges for LLM Adversarial Evaluation

Researchers evaluated whether fine-tuned encoder classifiers can effectively replace expensive LLM-based judges for detecting harmful outputs in large language models. The study benchmarked ModernBERT family encoders against LLM judges and rule-based methods across adversarial datasets, finding that encoders offer a cost- and latency-efficient alternative for safety evaluation in production environments.

🧠 Claude