#llm-alignment2 articles

2 articles

AINeutralarXiv – CS AI · 4h ago2

🧠

RewardUQ: A Unified Framework for Uncertainty-Aware Reward Models

Researchers introduce RewardUQ, a unified framework for evaluating uncertainty quantification in reward models used to align large language models with human preferences. The study finds that model size and initialization have the most significant impact on performance, while providing an open-source Python package to advance the field.

AIBullisharXiv – CS AI · 4h ago4

🧠

Alignment through Meta-Weighted Online Sampling: Bridging the Gap between Data Generation and Preference Optimization

Researchers propose MetaAPO, a new framework for aligning large language models with human preferences that dynamically balances online and offline training data. The method uses a meta-learner to evaluate when on-policy sampling is beneficial, resulting in better performance while reducing online annotation costs by 42%.