FraudBench: A Multimodal Benchmark for Detecting AI-Generated Fraudulent Refund Evidence

Researchers introduce FraudBench, a multimodal benchmark dataset designed to detect AI-generated fraudulent refund evidence in e-commerce, food delivery, and travel services. The study reveals that current AI detection systems struggle significantly with claim-conditioned fake-damage detection, with specialized detectors failing to reliably distinguish synthetic fraud from authentic evidence.

FraudBench addresses a critical vulnerability in digital commerce: the ability of bad actors to generate convincing synthetic images to support fraudulent refund claims. As generative AI models become increasingly sophisticated, the gap between what humans and machines can verify grows wider. The benchmark combines real-world evidence from actual e-commerce platforms with AI-synthesized fraud scenarios, creating a practical testing ground that moves beyond generic image authenticity classification to context-aware verification.

The research emerges amid a broader digital trust crisis where platforms struggle to authenticate user claims without manual intervention. Refund fraud costs e-commerce companies billions annually, and AI-generated visual evidence represents a new frontier for sophisticated abuse. The study's evaluation methodology—testing MLLMs, specialized detectors, and human judges simultaneously—reveals uncomfortable truths: current models achieve fake-damage detection rates below 50% on many generator subsets, while exhibiting high false-positive rates on genuinely damaged evidence.

This creates immediate operational challenges for platform operators. E-commerce, food delivery, and travel services face pressure to implement better verification mechanisms while maintaining user experience. The performance gap between specialized detectors and generic MLLMs suggests the market opportunity for domain-specific verification tools. However, the inconsistency across different synthesis models indicates no silver-bullet solution exists yet. Companies handling high-volume fraud detection will need hybrid approaches combining human review, detector ensembles, and behavioral signals.

Looking forward, the adversarial cycle will intensify as detection improves and attackers adapt. FraudBench's contribution lies in providing researchers a standardized evaluation framework to track this arms race systematically. Platform security teams should monitor detector development closely while implementing behavioral friction in refund workflows.

• →Current AI detection systems fail to identify AI-generated fraudulent refund evidence at reliable rates, with detection rates below 50% on many synthetic image generators.
• →FraudBench provides the first multimodal benchmark combining real e-commerce evidence with context-aware fraud detection evaluation across multiple service categories.
• →Specialized AI detectors outperform general MLLMs but produce high false-positive rates on genuine damaged evidence, creating operational friction.
• →The gap between synthetic fraud sophistication and detection capability poses growing financial risk for e-commerce, food delivery, and travel platforms.
• →Effective refund-fraud prevention requires hybrid verification approaches rather than relying on any single detection mechanism.