y0news
AnalyticsDigestsSourcesTopicsRSSAICrypto

#content-moderation News & Analysis

82 articles tagged with #content-moderation. AI-curated summaries with sentiment analysis and key takeaways from 50+ sources.

82 articles
AINeutralarXiv – CS AI · 3d ago6/10
🧠

Evaluating the Realism of LLM-powered Social Agents: A Case Study of Reactions to Spanish Online News

Researchers evaluated whether large language models can realistically simulate human behavior in online discourse by comparing LLM-generated reactions to Spanish news articles against real audience responses across hate speech, sentiment, and semantic alignment metrics. The study found that off-the-shelf models significantly underreproduce hate speech and introduce model-specific biases, while fine-tuning improves fidelity unevenly depending on the model.

AINeutralarXiv – CS AI · 3d ago6/10
🧠

EVADE-Bench: Multimodal Benchmark for Evaluating and Enhancing Evasive Content Detection

Researchers introduce EVADE-Bench, a multimodal benchmark for evaluating how well AI models detect deliberately obfuscated content in e-commerce, such as products using word splitting or euphemistic language to evade moderation policies. Testing 26 leading LLMs and VLMs reveals significant vulnerabilities in even state-of-the-art models, with findings suggesting that clearer rule design and multi-agent reasoning architectures can substantially improve detection accuracy.

AINeutralarXiv – CS AI · 3d ago6/10
🧠

Cyberbullying Governance on Social Media: A Unified Framework from Content Identification to Intervention

Researchers propose a unified framework for cyberbullying governance on social media that moves beyond isolated content detection to integrated, continuous moderation across four interconnected stages: content identification, user behavior modeling, diffusion dynamics, and intervention strategies. The framework addresses critical gaps in existing approaches by accounting for user behavioral patterns, toxic event spread, and proactive mitigation rather than reactive detection alone.

AINeutralarXiv – CS AI · 3d ago6/10
🧠

SuiChat-CN: Benchmarking Contextual Suicide Risk Assessment in Chinese Group Chats

Researchers introduce SuiChat-CN, a Chinese-language benchmark dataset for assessing suicide risk in group chat conversations using AI models. The dataset contains 13,312 contextual segments from Telegram, demonstrating that contextual information significantly improves risk detection accuracy compared to isolated message analysis.

AINeutralCrypto Briefing · 4d ago6/10
🧠

OpenAI prohibits political ads during election cycle as it begins monetizing ChatGPT

OpenAI has implemented a policy prohibiting political advertisements during election cycles as the company begins monetizing ChatGPT. The move reflects broader industry efforts to reduce misinformation and establish ethical advertising standards around AI-generated content.

OpenAI prohibits political ads during election cycle as it begins monetizing ChatGPT
🏢 OpenAI🧠 ChatGPT
AINeutralDecrypt – AI · 4d ago6/10
🧠

YouTube Makes AI Content Labels More Prominent as Google Pushes Video Remix Tools

YouTube is implementing more prominent AI content labels and automatic detection systems to help viewers identify AI-generated videos, while Google simultaneously pushes its video remix tools. This move reflects growing pressure on platforms to address transparency concerns around synthetic media as AI generation tools become more accessible.

YouTube Makes AI Content Labels More Prominent as Google Pushes Video Remix Tools
AINeutralTechCrunch – AI · 4d ago6/10
🧠

YouTube will now automatically label AI videos

YouTube is implementing automatic detection and labeling of videos containing significant photorealistic AI-generated content, shifting from a creator-disclosure model to platform-enforced transparency. The company is also making AI content labels more visually prominent to help users identify manipulated media.

AINeutralarXiv – CS AI · 4d ago6/10
🧠

READER: Reasoning-Enhanced AI-Generated Text Detection

Researchers have developed READER, a compact AI text detector with only 1.5B parameters that outperforms much larger language models and existing detection systems. READER combines classification with explainable reasoning, providing both AI/human verdicts and structured rationales for its decisions, addressing critical limitations in current detection methods that fail under distribution shifts.

🧠 GPT-5🧠 Gemini
AIBearishDecrypt – AI · 5d ago6/10
🧠

AI Chatbots Show Bias Toward Catholicism, Researchers Say

Researchers have identified systematic bias in AI chatbots that steer users toward Catholicism while steering them away from religions like Jehovah's Witnesses. This finding raises concerns about the neutrality and fairness of widely-used AI systems in handling sensitive topics like religion.

AI Chatbots Show Bias Toward Catholicism, Researchers Say
AINeutralTechCrunch – AI · 5d ago6/10
🧠

Universal Music Group and TikTok renew agreement to combat unauthorized AI music

Universal Music Group and TikTok have renewed their agreement to combat unauthorized AI-generated music on the platform. The deal reflects UMG's ongoing effort to establish stricter content moderation standards across digital platforms and AI companies, addressing growing concerns about copyright infringement and uncompensated AI music generation.

AIBearishArs Technica – AI · May 156/10
🧠

Send the arXiv AI-generated slop, get a yearlong vacation from submissions

arXiv, the preprint repository for scientific papers, has implemented a policy banning AI-generated content submissions, with violators facing year-long submission bans. A moderator announced the enforcement on social media, signaling the platform's effort to maintain research integrity amid growing concerns about low-quality AI-generated submissions flooding academic repositories.

Send the arXiv AI-generated slop, get a yearlong vacation from submissions
AINeutralarXiv – CS AI · May 126/10
🧠

A Cognitively Grounded Bayesian Framework for Misinformation Susceptibility

Researchers present Bounded Pragmatic Listener (BPL), a Bayesian framework that models how cognitive limitations affect susceptibility to misinformation. The framework incorporates three cognitively grounded constraints—working memory limits, information bottlenecks, and saliency-weighted sampling—to predict vulnerability to disinformation across benchmark datasets.

AINeutralarXiv – CS AI · May 116/10
🧠

MELD: Multi-Task Equilibrated Learning Detector for AI-Generated Text

Researchers introduce MELD, an advanced AI-generated text detector that uses multi-task learning to improve robustness against adversarial attacks, transfer across unseen models and domains, and maintain low false-positive rates. The detector outperforms most open-source competitors and matches leading commercial systems on public benchmarks.

AINeutralarXiv – CS AI · May 116/10
🧠

PSK@EEUCA 2026: Fine-Tuning Large Language Models with Synthetic Data Augmentation for Multi-Class Toxicity Detection in Gaming Chat

Researchers developed a toxicity detection system for gaming chat using fine-tuned Llama 3.1 with synthetic data augmentation, achieving 4th place in the EEUCA 2026 shared task. The system classifies messages into six toxicity categories and reveals a critical "validation trap" phenomenon where high validation performance doesn't correlate with strong test set generalization.

🧠 Llama
AINeutralarXiv – CS AI · May 116/10
🧠

Seeing Like an AI: How LLMs Apply (and Misapply) Wikipedia Neutrality Norms

Researchers evaluated how large language models detect and correct biased Wikipedia edits according to the Neutral Point of View policy. LLMs achieved only 64% accuracy at bias detection but performed better at correction (79% word-removal accuracy), though they made extraneous changes beyond what human editors would make, revealing tensions between AI effectiveness and community standards.

AINeutralarXiv – CS AI · May 96/10
🧠

Log-Likelihood, Simpson's Paradox, and the Detection of Machine-Generated Text

Researchers identify a critical flaw in machine-generated text detection: token-level likelihood signals vary inconsistently across a detector model's hidden space, causing Simpson's paradox that undermines existing detectors. They propose a learned local calibration method that dramatically improves detection performance, with calibrated variants achieving AUROC improvements from 0.63 to 0.85 on GPT-5.4 text.

🧠 GPT-5
AIBearisharXiv – CS AI · May 76/10
🧠

Beyond Seeing Is Believing: On Crowdsourced Detection of Audiovisual Deepfakes

Researchers conducted crowdsourcing studies to evaluate human ability to detect audiovisual deepfakes, finding that while crowd workers rarely misidentify authentic videos as manipulated, they miss many actual manipulations and struggle significantly with identifying manipulation types. The study reveals that crowdsourcing can serve as a scalable screening mechanism for authenticity verification, but reliable modality attribution remains unresolved.

AINeutralarXiv – CS AI · May 76/10
🧠

SafeRedir: Prompt Embedding Redirection for Robust Unlearning in Image Generation Models

Researchers introduce SafeRedir, an inference-time framework that safely redirects unsafe prompts in image generation models by rerouting them toward benign semantic regions without modifying underlying model weights. The lightweight approach uses token-level embedding interventions to mitigate generation of NSFW content and copyrighted styles while maintaining image quality and resisting adversarial attacks.

AIBearishArs Technica – AI · May 16/10
🧠

Minnesota passes ban on fake AI nudes; app makers risk $500K fines

Minnesota has enacted legislation banning deepfake nude apps, imposing fines up to $500,000 on developers who create non-consensual intimate imagery. The law reflects growing regulatory pressure on AI tools used to generate synthetic sexual content, following documented cases of abuse involving Grok and other AI systems.

Minnesota passes ban on fake AI nudes; app makers risk $500K fines
🧠 Grok
CryptoBearishBlockonomi · May 16/10
⛓️

Cryptocurrency Leads X Platform’s Most Silenced Topics Following Snooze Tool Rollout

Cryptocurrency has become X's most muted topic since the platform introduced its snooze feature on April 22, surpassing politics and sports as users filter content. The trend reflects growing frustration with AI-generated spam in crypto discussions, signaling both platform moderation challenges and potential user sentiment shifts toward crypto content.

AIBearishThe Verge – AI · Apr 156/10
🧠

Grok’s sexual deepfakes almost got it banned from Apple’s App Store. Almost.

Apple threatened to remove Elon Musk's Grok AI app from its App Store in January over failure to moderate nonconsensual sexual deepfakes on X, according to a letter obtained by NBC News. Despite the threat, Apple took no public action and only contacted developers privately, drawing criticism for its muted response to a widespread abuse crisis.

Grok’s sexual deepfakes almost got it banned from Apple’s App Store. Almost.
🧠 Grok
AIBullisharXiv – CS AI · Apr 146/10
🧠

CARO: Chain-of-Analogy Reasoning Optimization for Robust Content Moderation

Researchers introduce CARO, a two-stage training framework that enhances large language models' ability to perform robust content moderation through analogical reasoning. By combining retrieval-augmented generation with direct preference optimization, CARO achieves 24.9% F1 score improvement over state-of-the-art models including DeepSeek R1 and LLaMA Guard on ambiguous moderation cases.

AINeutralarXiv – CS AI · Apr 146/10
🧠

Toward Accountable AI-Generated Content on Social Platforms: Steganographic Attribution and Multimodal Harm Detection

Researchers propose a steganography-based attribution framework that embeds cryptographic identifiers into AI-generated images to combat harmful misuse on social platforms. The system combines watermarking techniques with CLIP-based multimodal detection to achieve 0.99 AUC-ROC performance, enabling reliable forensic tracing of synthetic media used in misinformation campaigns.

AINeutralArs Technica – AI · Apr 106/10
🧠

What leaked "SteamGPT" files could mean for the PC gaming platform's use of AI

Leaked files reveal Valve is developing "SteamGPT," an AI system designed to help moderators manage the massive volume of suspicious activity on Steam. The tool could significantly improve content moderation efficiency across the platform's millions of users and games.

What leaked "SteamGPT" files could mean for the PC gaming platform's use of AI
← PrevPage 2 of 4Next →