AIBullisharXiv – CS AI · 6h ago7/10
🧠
From "Weak" Signals to Strong Models: Preference Delta Aggregation with LoRA Merging
Researchers propose Preference Delta Aggregation (PDA), a framework that combines weak preference signals from multiple smaller language model pairs into LoRA adapters, then merges them using Geometric Alignment Merging to improve larger models. The approach achieves 6.8-7.3 point improvements on knowledge reasoning and agentic search benchmarks by effectively composing complementary capabilities.