23 articles tagged with #human-feedback. AI-curated summaries with sentiment analysis and key takeaways from 50+ sources.
AINeutralarXiv – CS AI · 2d ago7/10
🧠Researchers introduce WIMHF, a method using sparse autoencoders to decode what human feedback datasets actually measure and express about AI model preferences. The technique identifies interpretable features across 7 datasets, revealing diverse preference patterns and uncovering potentially unsafe biases—such as LMArena users voting against safety refusals—while enabling targeted data curation that improved safety by 37%.
AIBullisharXiv – CS AI · Mar 66/10
🧠Researchers propose VISA (Value Injection via Shielded Adaptation), a new framework for aligning Large Language Models with human values while avoiding the 'alignment tax' that causes knowledge drift and hallucinations. The system uses a closed-loop architecture with value detection, translation, and rewriting components, demonstrating superior performance over standard fine-tuning methods and GPT-4o in maintaining factual consistency.
🧠 GPT-4
AIBullisharXiv – CS AI · Mar 56/10
🧠Researchers propose a new framework called Critic Rubrics to bridge the gap between academic coding agent benchmarks and real-world applications. The system learns from sparse, noisy human interaction data using 24 behavioral features and shows significant improvements in code generation tasks including 15.9% better reranking performance on SWE-bench.
AIBullisharXiv – CS AI · Mar 47/104
🧠Researchers present a new mathematical framework for training AI reward models using Likert scale preferences instead of simple binary comparisons. The approach uses ordinal regression to better capture nuanced human feedback, outperforming existing methods across chat, reasoning, and safety benchmarks.
AIBullisharXiv – CS AI · Mar 46/102
🧠Researchers developed a two-stage learning framework enabling robots to perform complex manipulation tasks like food peeling with over 90% success rates. The system combines force-aware imitation learning with human preference-based refinement, achieving strong generalization across different produce types using only 50-200 training examples.
AIBullishOpenAI News · Jan 277/107
🧠OpenAI has developed InstructGPT models that significantly improve upon GPT-3's ability to follow user instructions while being more truthful and less toxic. These models use human feedback training and alignment research techniques, and have been deployed as the default language models on OpenAI's API.
AIBullishOpenAI News · Sep 47/105
🧠Researchers have successfully applied reinforcement learning from human feedback (RLHF) to improve language model summarization capabilities. This approach uses human preferences to guide the training process, resulting in models that produce higher quality summaries aligned with human expectations.
AIBullishOpenAI News · Jun 137/107
🧠OpenAI and DeepMind have collaborated to develop an algorithm that can learn human preferences by comparing two proposed behaviors, eliminating the need for humans to manually write goal functions. This approach aims to reduce dangerous AI behavior that can result from oversimplified or incorrect goal specifications.
AINeutralarXiv – CS AI · 2d ago6/10
🧠Researchers introduce SciTune, a framework for fine-tuning large language models with human-curated scientific multimodal instructions from academic publications. The resulting LLaMA-SciTune model demonstrates superior performance on scientific benchmarks compared to state-of-the-art alternatives, with results suggesting that high-quality human-generated data outweighs the volume advantage of synthetic training data for specialized scientific tasks.
AINeutralarXiv – CS AI · 2d ago6/10
🧠Researchers demonstrate that human preferences can be influenced to better align with the mathematical models used in RLHF algorithms, without changing underlying reward functions. Through three interventions—revealing model parameters, training humans on preference models, and modifying elicitation questions—the study shows significant improvements in preference data quality and AI alignment outcomes.
AIBullisharXiv – CS AI · Apr 66/10
🧠Researchers have developed OPRIDE, a new algorithm for offline preference-based reinforcement learning that significantly improves query efficiency. The algorithm addresses key challenges of inefficient exploration and overoptimization through principled exploration strategies and discount scheduling mechanisms.
AIBullisharXiv – CS AI · Mar 96/10
🧠PRISM is a new AI method that combines imitation learning and reinforcement learning to train robotic manipulation systems using human instructions and feedback. The approach allows generic robotic policies to be refined for specific tasks through natural language descriptions and human corrections, improving performance in pick-and-place tasks while reducing computational requirements.
AINeutralarXiv – CS AI · Mar 96/10
🧠A systematic literature review of 346 papers reveals critical flaws in AI data annotation practices, arguing that treating human disagreement as 'noise' rather than meaningful signal undermines model quality. The study proposes pluralistic annotation frameworks that embrace diverse human perspectives instead of forcing artificial consensus.
AINeutralarXiv – CS AI · Mar 27/1017
🧠Researchers propose a unified theory explaining why AI models trained on human feedback exhibit persistent error floors that cannot be eliminated through scaling alone. The study demonstrates that human supervision acts as an information bottleneck due to annotation noise, subjective preferences, and language limitations, requiring auxiliary non-human signals to overcome these structural limitations.
AINeutralarXiv – CS AI · Mar 26/1010
🧠Researchers introduce RewardUQ, a unified framework for evaluating uncertainty quantification in reward models used to align large language models with human preferences. The study finds that model size and initialization have the most significant impact on performance, while providing an open-source Python package to advance the field.
AINeutralarXiv – CS AI · Feb 276/104
🧠Researchers propose using psychometric modeling to correct systematic biases in human evaluations of AI systems, demonstrating how Item Response Theory can separate true AI output quality from rater behavior inconsistencies. The approach was tested on OpenAI's summarization dataset and showed improved reliability in measuring AI model performance.
AINeutralarXiv – CS AI · Feb 275/107
🧠Researchers conducted a cross-modal study comparing human preference annotations between text and audio formats for AI alignment. The study found that while audio preferences are as reliable as text, different modalities lead to different judgment patterns, with synthetic ratings showing promise as replacements for human annotations.
$NEAR
AINeutralOpenAI News · Aug 246/107
🧠An AI research organization outlines their approach to alignment research, focusing on improving AI systems' ability to learn from human feedback and assist in AI evaluation. Their ultimate goal is developing a sufficiently aligned AI system capable of solving all remaining AI alignment challenges.
AINeutralOpenAI News · Sep 235/105
🧠This article discusses scaling human oversight of AI systems for tasks that are difficult to evaluate, specifically focusing on summarizing books with human feedback. The approach addresses the challenge of maintaining human control and evaluation in AI applications where traditional assessment methods may be insufficient.
AINeutralOpenAI News · Sep 196/106
🧠OpenAI successfully fine-tuned a 774M parameter GPT-2 model using human feedback for tasks like summarization and text continuation. The research revealed challenges where human labelers' preferences didn't align with developers' intentions, with summarization models learning to copy text wholesale rather than generate original summaries.
AINeutralOpenAI News · Aug 35/107
🧠RL-Teacher is an open-source implementation that enables AI training through occasional human feedback instead of traditional hand-crafted reward functions. This technique was developed as a step toward creating safer AI systems and addresses reinforcement learning challenges where rewards are difficult to specify.
AINeutralarXiv – CS AI · Mar 274/10
🧠Researchers used eye-tracking to analyze how humans make preference judgments when evaluating AI-generated images, finding that gaze patterns can predict both user choices and confidence levels. The study revealed that participants' eyes shift toward chosen images about one second before making decisions, and gaze features achieved 68% accuracy in predicting binary choices.
AINeutralHugging Face Blog · Dec 91/106
🧠The article appears to be about Reinforcement Learning from Human Feedback (RLHF), a machine learning technique used to train AI models based on human preferences and feedback. However, no article body content was provided for analysis.
