DeFrame: Debiasing Large Language Models Against Framing Effects

Researchers identify 'framing disparity' as a hidden source of bias in large language models, where semantically equivalent prompts expressed differently produce inconsistent fairness outcomes. The study proposes DeFrame, a debiasing method that improves LLM consistency across alternative framings, addressing a gap between standard fairness evaluations and real-world performance.

The research tackles a nuanced but consequential problem in AI fairness: LLMs can appear unbiased under controlled testing conditions yet fail when confronted with semantically identical requests phrased differently. This framing effect—the tendency of systems to respond differently to 'A is better than B' versus 'B is worse than A'—reveals fragility in current fairness benchmarks that researchers and developers have largely overlooked. The discovery matters because real-world deployment exposes models to countless phrasings beyond standardized evaluation datasets, creating hidden vulnerabilities that persist despite existing debiasing efforts.

Existing debiasing methods, while improving average fairness scores, leave these framing-induced disparities unresolved. The paper's key contribution is demonstrating that consistency across framings must become an explicit debiasing objective, not merely a side effect. DeFrame enforces this by training models to produce invariant responses regardless of how requests are linguistically framed, effectively hardening fairness properties against prompt variation.

For AI practitioners and enterprises deploying LLMs in sensitive applications—hiring, lending, content moderation—this research signals that fairness audits require more sophisticated methodology. Current benchmarking practices miss systematic vulnerabilities. Organizations must either implement framing-aware evaluation protocols or adopt debiasing approaches like DeFrame to reduce litigation and reputational risk. This work elevates fairness engineering from pass-fail testing to robustness validation, pushing the industry toward production-ready AI systems that maintain fair behavior across diverse input expressions.

• →LLMs show significant fairness variation across semantically equivalent prompts, indicating hidden bias that standard evaluations miss.
• →Existing debiasing methods improve average fairness but fail to reduce disparities caused by different prompt framings.
• →DeFrame enforces consistency across alternative framings to produce more robust and fair LLM responses.
• →Framing disparities represent a critical gap between controlled fairness benchmarks and real-world deployment conditions.
• →Organizations deploying LLMs for sensitive decisions need framing-aware evaluation protocols to identify and mitigate hidden biases.