6 articles tagged with #preference-alignment. AI-curated summaries with sentiment analysis and key takeaways from 50+ sources.
Researchers introduce Multi-Objective Control (MOC), a new approach that trains a single large language model to generate personalized responses based on individual user preferences across multiple objectives. The method uses multi-objective optimization principles in reinforcement learning from human feedback to create more controllable and adaptable AI systems.
Researchers propose an improved Nash Learning from Human Feedback (NLHF) algorithm that addresses exploration challenges in preference alignment for large language models. The new method achieves better regret bounds without exponential dependence on regularization parameters and demonstrates empirical improvements when fine-tuning Llama-3-8B.
Researchers propose In-Context Reward Adaptation, a transformer-based framework that dynamically models diverse human preferences without costly retraining. By incorporating human response time as an auxiliary signal, the approach enables language models to adapt to unseen preference domains on-the-fly, addressing a critical limitation of static reward models used in RLHF systems.
Researchers introduce StoryLens, a framework for preference-aligned story rewriting that goes beyond style transfer to incorporate context-aware narrative enrichment. Human studies show context-enhanced rewriting improves reader satisfaction by 24.5% compared to style-only approaches, supported by a new benchmark, reward model, and two-stage rewriting system combining supervised learning with reinforcement learning.
Researchers introduce Open-Universe Assistance Games (OU-AGs), a framework enabling LLM-based agents to infer and align with human preferences through open-ended dialogue. The GOOD method extracts evolving goals from natural language interactions using probabilistic inference, demonstrating improved user intent alignment across shopping, robotics, and coding domains without requiring large offline datasets.
Researchers developed a framework to make large language model-based query expansion more efficient by distilling knowledge from powerful teacher models into compact student models. The approach uses retrieval feedback and preference alignment to maintain 97% of the original performance while dramatically reducing inference costs.
