#preference-data News & Analysis

2 articles tagged with #preference-data. AI-curated summaries with sentiment analysis and key takeaways from 50+ sources.

2 articles

AIBearisharXiv – CS AI · 4d ago7/10

🧠

Elias in the Lighthouse, Again? Diagnosing Low Diversity in LLM Stories

Researchers found that LLM-generated stories suffer from severe lack of diversity, with just 11 specific words appearing in 88.3% of outputs across multiple models. These recurring elements—character names like Elias and Mara, settings like lighthouses, and professions like clockmaker—originate from preference data used in model alignment rather than training data, revealing how small datasets can disproportionately shape AI outputs.

AINeutralarXiv – CS AI · Apr 147/10

🧠

What's In My Human Feedback? Learning Interpretable Descriptions of Preference Data

Researchers introduce WIMHF, a method using sparse autoencoders to decode what human feedback datasets actually measure and express about AI model preferences. The technique identifies interpretable features across 7 datasets, revealing diverse preference patterns and uncovering potentially unsafe biases—such as LMArena users voting against safety refusals—while enabling targeted data curation that improved safety by 37%.