2 articles tagged with #dataset-audit. AI-curated summaries with sentiment analysis and key takeaways from 50+ sources.
Researchers auditing 39 deepfake speech detection datasets found critical flaws undermining fairness claims and generalization metrics. Most datasets lack demographic metadata, and widespread overlap in underlying training sources creates illusions of robustness that may not transfer to real-world scenarios.
Researchers audited the MedCalc-Bench benchmark for evaluating AI models on clinical calculator tasks, finding over 20 errors in the dataset and showing that simple 'open-book' prompting achieves 81-85% accuracy versus previous best of 74%. The study suggests the benchmark measures formula memorization rather than clinical reasoning, challenging how AI medical capabilities are evaluated.
