AINeutralarXiv – CS AI · 4h ago6/10
🧠
Polynomial Expansion Rank Adaptation: Enhancing Low-Rank Fine-Tuning with High-Order Interactions
Researchers propose Polynomial Expansion Rank Adaptation (PERA), a novel fine-tuning method that enhances Low-Rank Adaptation (LoRA) by incorporating high-order polynomial interactions into low-rank factors. PERA improves the expressive capacity of LLM fine-tuning without increasing computational costs, demonstrating consistent performance gains across benchmarks while maintaining the efficiency benefits of rank-constrained adaptation.