Taxonomy of Risks on Automated Fact-Checking Systems Considering its Propagation

Researchers have identified 32 specific risks in automated fact-checking systems that use AI and large language models, focusing on how errors propagate from initial risk factors through hazardous situations to eventual harm. The study demonstrates that traditional IT security assessment methods like STRIDE fail to capture emerging risks unique to automated fact-checking systems, highlighting critical gaps in safeguarding these tools against spreading misinformation.

The proliferation of fake news on social media has created urgent demand for automated fact-checking solutions, yet these systems introduce their own risks that conventional security frameworks struggle to address. Researchers have systematized 32 distinct risks by modeling a three-stage propagation pathway: risk factors (root causes), hazardous situations (unsafe states), and harm (actual damage). This taxonomy reveals that errors in automated systems can amplify rather than mitigate misinformation, potentially causing defamation or further disinformation spread when incorrect fact-check verdicts reach millions of users simultaneously.

The findings emerge as AI-powered fact-checking gains adoption among platforms and organizations seeking to scale moderation beyond human capacity. Traditional security methods like STRIDE focus on intentional attacks and system vulnerabilities but don't adequately capture risks from AI model failures, training data biases, or prompt injection attacks specific to language models. The research validates this gap by showing their guide-word approach identifies risks that STRIDE misses entirely.

For stakeholders deploying automated fact-checking, this research signals the need for domain-specific risk frameworks before systems launch at scale. Organizations face potential liability if automated verdicts cause measurable harm, particularly where defamation claims arise. The implication extends beyond content moderation: as AI systems make high-stakes decisions across journalism, law, and finance, understanding failure modes becomes commercially and legally essential. Companies investing in fact-checking infrastructure should demand risk assessments using methodologies specifically designed for AI systems rather than generic IT security protocols.

• →Automated fact-checking systems powered by AI create 32 identified risk categories across three propagation stages from root causes to user-facing harm.
• →Conventional IT security assessment methods like STRIDE fail to detect AI-specific risks in fact-checking systems, leaving critical vulnerabilities unaddressed.
• →Incorrect automated fact-checks can amplify misinformation at scale, potentially causing defamation and eroding public trust in fact-checking institutions.
• →Organizations deploying automated fact-checking face legal liability and reputational damage if systems produce harmful incorrect verdicts.
• →Domain-specific risk assessment frameworks for AI systems are necessary before automated fact-checking tools are deployed in production environments.