AINeutralarXiv – CS AI · 10h ago7/10
🧠
HALAS: A Human-Annotated Dataset of Hallucinations of Modern ASR Systems
Researchers introduce HALAS, the first human-annotated dataset documenting naturally occurring hallucinations from seven state-of-the-art ASR systems on real earnings call recordings. The benchmark reveals that hallucinations persist even in nearly correct transcriptions and establishes rigorous evaluation methods, with current detection techniques achieving only 53.1% F1 scores despite character-level metrics reaching 81% ROC-AUC.