#annotation News & Analysis

4 articles tagged with #annotation. AI-curated summaries with sentiment analysis and key takeaways from 50+ sources.

4 articles

AINeutralarXiv – CS AI · 4d ago6/10

🧠

Sign-Language Datasets at Scale: A Comprehensive Survey on Resources, Benchmarks, and Annotation Standards

Researchers have conducted a comprehensive survey of 120 sign-language datasets across 35 languages, identifying critical gaps in annotation standards, linguistic coverage, and real-world applicability. The study introduces a standardized 24-field datasheet and open-source documentation framework to improve dataset quality and advance accessibility technologies for Deaf and Hard-of-Hearing communities.

AINeutralarXiv – CS AI · May 116/10

🧠

Active teacher selection for reward learning

Researchers introduce the Hidden Utility Bandit (HUB) framework to address a critical limitation in reward learning systems: their reliance on feedback from a single idealized teacher. The framework models teacher heterogeneity in rationality, expertise, and cost, enabling Active Teacher Selection (ATS) algorithms that strategically choose which teachers to query, demonstrating superior performance in paper recommendation and vaccine testing applications.

AINeutralarXiv – CS AI · May 96/10

🧠

Understanding Annotator Safety Policy with Interpretability

Researchers introduce Annotator Policy Models (APMs), interpretable machine learning models that extract and visualize annotators' implicit safety policies from labeling behavior alone. By revealing disagreement sources—operational failures, policy ambiguity, and value pluralism—APMs enable more transparent and inclusive AI safety policy design without requiring costly additional annotation.

AIBearisharXiv – CS AI · Mar 36/104

🧠

Are LLMs Ready to Replace Bangla Annotators?

A comprehensive study of 17 Large Language Models as automated annotators for Bangla hate speech detection reveals significant bias and instability issues. The research found that larger models don't necessarily perform better than smaller, task-specific ones, raising concerns about LLM reliability for sensitive annotation tasks in low-resource languages.