LoRA-DA: Data-Aware Initialization for Low-Rank Adaptation via Asymptotic Analysis

Researchers introduce LoRA-DA, a new initialization method for Low-Rank Adaptation that leverages target-domain data and theoretical optimization principles to improve fine-tuning performance. The method outperforms existing initialization approaches across multiple benchmarks while maintaining computational efficiency.