AIBullisharXiv – CS AI · 8h ago7/10
🧠
Litmus: Zero-Label, Code-Driven Metric Specification for Evaluating AI Systems
Researchers introduce Litmus, a zero-label evaluation system that automatically designs metrics for AI pipelines by analyzing source code rather than relying on manual labeling. The system identifies what needs to be measured and why before constructing justified metric portfolios, outperforming existing baselines on three real-world AI applications including financial and scientific tasks.