PICACO: Pluralistic In-Context Value Alignment of LLMs via Total Correlation Optimization

Researchers introduce PICACO, a novel in-context alignment method that optimizes meta-instructions to help large language models better understand and balance multiple, often conflicting human values without fine-tuning. The approach uses total correlation optimization to improve alignment across up to 8 distinct values while reducing noise, addressing a key limitation where LLMs struggle to reconcile competing preferences in single prompts.

PICACO represents a meaningful advancement in aligning LLMs with pluralistic human values, a challenge that has grown increasingly relevant as AI systems enter diverse global markets. Traditional in-context alignment methods operate under a fundamental constraint—they treat value alignment as a binary task where LLMs either satisfy or fail to satisfy stated preferences. The paper identifies a critical gap: humans hold competing values (stimulation versus tradition, individual freedom versus collective welfare), and current prompt-based alignment techniques cannot simultaneously optimize for these tensions without degrading performance on one or more dimensions.

The research addresses this through a mathematically grounded approach using total correlation optimization, which strengthens the statistical relationship between specified values and model outputs while filtering out irrelevant noise. This method sidesteps the computational expense of fine-tuning while maintaining compatibility with both proprietary and open-source models, making it practically deployable across diverse LLM ecosystems.

The implications extend beyond academic interest. For developers and AI companies, PICACO offers a lightweight technique to improve model robustness across different value frameworks without retraining—critical for serving international markets with conflicting regulatory and cultural requirements. The ability to balance multiple values simultaneously reduces the risk of unintended model biases that emerge when optimizing for single objectives. For enterprise users, this means more reliable alignment with nuanced organizational values and reduced need for expensive post-training customization. The research validates performance across five value sets with notable improvements over existing baselines, suggesting the approach generalizes reasonably well.

• →PICACO optimizes meta-instructions to resolve value conflicts without fine-tuning, addressing the Instruction Bottleneck challenge in in-context alignment.
• →The method uses total correlation optimization to strengthen relationships between specified values and LLM outputs while reducing noise.
• →Achieves balanced performance across up to 8 distinct values, outperforming recent alignment baselines on multiple value sets.
• →Works with both black-box and open-source LLMs, making it broadly applicable across different model architectures.
• →Offers cost-effective alignment for enterprises requiring pluralistic value representation without expensive model retraining.