AINeutralarXiv – CS AI · 7h ago7/10
🧠
Hidden Consensus:Preference-Validity Compression in Human Feedback
Researchers identify a critical flaw in standard RLHF (Reinforcement Learning from Human Feedback) pipelines: they collapse culturally and contextually diverse human preferences into single scalar rewards, potentially misaligning AI systems in pluralistic societies. A study of Malaysian annotators found that 79% of prompts contained multiple majority-supported valid responses that standard aggregation would discard, suggesting current alignment measurement fails to capture legitimate interpretive diversity.