Metamorphic Testing with the Rashomon Set: Explanation Faithfulness in Machine Learning
Researchers propose a metamorphic testing framework to evaluate the trustworthiness of machine learning model explanations by identifying inconsistencies between model predictions and feature attributions, addressing the Rashomon effect where multiple models achieve similar performance but yield conflicting explanations.
The research addresses a critical challenge in machine learning interpretability: the Rashomon effect, where functionally equivalent models produce fundamentally different explanations. This phenomenon undermines confidence in popular explanation methods like SHAP and LIME, which practitioners rely on for model validation and decision-making. The proposed metamorphic testing framework establishes five consistency relations that validate whether feature attributions genuinely reflect model behavior without requiring ground-truth labels—a significant practical advantage since labeled data remains expensive and scarce.
This work emerges from growing recognition that explainability alone doesn't guarantee trustworthiness. Financial institutions, healthcare systems, and regulatory bodies increasingly demand verifiable explanations for high-stakes predictions. Prior approaches focused on comparing explanations across models or validating against known patterns, but this framework innovates by testing internal consistency between prediction behavior and attribution patterns.
The implications extend across industries deploying ML for critical decisions. Organizations using SHAP or LIME for compliance, risk assessment, or stakeholder transparency now have a model-agnostic validation mechanism. This becomes particularly valuable in regulated sectors where explanation quality directly impacts regulatory approval and legal defensibility.
The framework's effectiveness on tabular regression datasets suggests immediate applicability to finance, healthcare, and supply chain domains. However, extension to unstructured data (images, text, time-series) and deep learning architectures remains an open question. Success here could reshape how organizations conduct model selection, moving beyond accuracy metrics toward holistic trustworthiness assessment.
- →Metamorphic testing provides a label-free method to assess explanation faithfulness across multiple ML models
- →The Rashomon effect challenges the reliability of post-hoc explanation methods commonly used in production systems
- →Framework validation focuses on consistency between model predictions and feature attributions rather than external ground truth
- →Model-agnostic approach enables broad adoption across different architectures and domains
- →Tool supports regulatory compliance and stakeholder trust by providing verifiable explanation quality metrics