38 articles tagged with #human-feedback. AI-curated summaries with sentiment analysis and key takeaways from 50+ sources.
Researchers have developed OPRIDE, a new algorithm for offline preference-based reinforcement learning that significantly improves query efficiency. The algorithm addresses key challenges of inefficient exploration and overoptimization through principled exploration strategies and discount scheduling mechanisms.
PRISM is a new AI method that combines imitation learning and reinforcement learning to train robotic manipulation systems using human instructions and feedback. The approach allows generic robotic policies to be refined for specific tasks through natural language descriptions and human corrections, improving performance in pick-and-place tasks while reducing computational requirements.
A systematic literature review of 346 papers reveals critical flaws in AI data annotation practices, arguing that treating human disagreement as 'noise' rather than meaningful signal undermines model quality. The study proposes pluralistic annotation frameworks that embrace diverse human perspectives instead of forcing artificial consensus.
Researchers propose a unified theory explaining why AI models trained on human feedback exhibit persistent error floors that cannot be eliminated through scaling alone. The study demonstrates that human supervision acts as an information bottleneck due to annotation noise, subjective preferences, and language limitations, requiring auxiliary non-human signals to overcome these structural limitations.
Researchers introduce RewardUQ, a unified framework for evaluating uncertainty quantification in reward models used to align large language models with human preferences. The study finds that model size and initialization have the most significant impact on performance, while providing an open-source Python package to advance the field.
Researchers propose using psychometric modeling to correct systematic biases in human evaluations of AI systems, demonstrating how Item Response Theory can separate true AI output quality from rater behavior inconsistencies. The approach was tested on OpenAI's summarization dataset and showed improved reliability in measuring AI model performance.
Researchers conducted a cross-modal study comparing human preference annotations between text and audio formats for AI alignment. The study found that while audio preferences are as reliable as text, different modalities lead to different judgment patterns, with synthetic ratings showing promise as replacements for human annotations.
An AI research organization outlines their approach to alignment research, focusing on improving AI systems' ability to learn from human feedback and assist in AI evaluation. Their ultimate goal is developing a sufficiently aligned AI system capable of solving all remaining AI alignment challenges.
This article discusses scaling human oversight of AI systems for tasks that are difficult to evaluate, specifically focusing on summarizing books with human feedback. The approach addresses the challenge of maintaining human control and evaluation in AI applications where traditional assessment methods may be insufficient.
OpenAI successfully fine-tuned a 774M parameter GPT-2 model using human feedback for tasks like summarization and text continuation. The research revealed challenges where human labelers' preferences didn't align with developers' intentions, with summarization models learning to copy text wholesale rather than generate original summaries.
RL-Teacher is an open-source implementation that enables AI training through occasional human feedback instead of traditional hand-crafted reward functions. This technique was developed as a step toward creating safer AI systems and addresses reinforcement learning challenges where rewards are difficult to specify.
Researchers used eye-tracking to analyze how humans make preference judgments when evaluating AI-generated images, finding that gaze patterns can predict both user choices and confidence levels. The study revealed that participants' eyes shift toward chosen images about one second before making decisions, and gaze features achieved 68% accuracy in predicting binary choices.
The article appears to be about Reinforcement Learning from Human Feedback (RLHF), a machine learning technique used to train AI models based on human preferences and feedback. However, no article body content was provided for analysis.
