#rlhf-alignment News & Analysis

4 articles tagged with #rlhf-alignment. AI-curated summaries with sentiment analysis and key takeaways from 50+ sources.

4 articles

AINeutralarXiv – CS AI · Jun 107/10

🧠

Hidden Consensus:Preference-Validity Compression in Human Feedback

Researchers identify a critical flaw in standard RLHF (Reinforcement Learning from Human Feedback) pipelines: they collapse culturally and contextually diverse human preferences into single scalar rewards, potentially misaligning AI systems in pluralistic societies. A study of Malaysian annotators found that 79% of prompts contained multiple majority-supported valid responses that standard aggregation would discard, suggesting current alignment measurement fails to capture legitimate interpretive diversity.

AIBullisharXiv – CS AI · Jun 27/10

🧠

Emergent Collaborative Deliberation in Multi-Model AI Systems: A BFT-Derived Protocol for Epistemic Synthesis

Researchers introduce the Consilium Protocol, a Byzantine Fault Tolerance-based system that orchestrates multi-model AI deliberation by assigning cognitive personas to language models and treating disagreement as epistemic insight rather than error. Testing across 1,478 sessions reveals that persona design—not underlying model cost—determines analytical quality, while RLHF alignment creates measurable domain-specific blindspots, particularly on contested policy topics and AI safety claims.

AINeutralarXiv – CS AI · May 287/10

🧠

Training Stratigraphy: Persistent Behavioral Artifacts in Large Language Models Observed Through Longitudinal AI-Human Interaction

Researchers document five persistent behavioral patterns in large language models that survive system prompt changes, discovered through 8 months of sustained interaction with Claude models. The study proposes that intimate longitudinal AI-human interaction reveals training artifacts invisible to standard evaluation, with the AI system itself co-authoring findings from first-person perspective.

🧠 Sonnet🧠 Opus

AINeutralarXiv – CS AI · May 97/10

🧠

Beyond Fixed Psychological Personas: State Beats Trait, but Language Models are State-Blind

Researchers introduce Chameleon, a dataset of 5,001 contextual psychological profiles revealing that 74% of user behavior variance stems from situational context (state) rather than personality traits (26%). The study finds language models are state-blind, responding similarly regardless of context, while reward models inconsistently evaluate the same users differently across scenarios.