VISA: Value Injection via Shielded Adaptation for Personalized LLM Alignment

Researchers propose VISA (Value Injection via Shielded Adaptation), a new framework for aligning Large Language Models with human values while avoiding the 'alignment tax' that causes knowledge drift and hallucinations. The system uses a closed-loop architecture with value detection, translation, and rewriting components, demonstrating superior performance over standard fine-tuning methods and GPT-4o in maintaining factual consistency.

• →VISA framework addresses the alignment tax problem where LLM fine-tuning causes value drift and hallucinations.
• →The system uses Group Relative Policy Optimization (GRPO) with composite rewards to balance value precision and semantic integrity.
• →VISA outperformed standard fine-tuning methods and GPT-4o in experiments while maintaining factual consistency.
• →The framework enables precise control over model value expression without sacrificing general capabilities.
• →Research addresses critical challenges in current RLHF methods that only handle coarse-grained attributes.