#parameter-efficient-finetuning News & Analysis

3 articles tagged with #parameter-efficient-finetuning. AI-curated summaries with sentiment analysis and key takeaways from 50+ sources.

3 articles

AIBullisharXiv – CS AI · May 16/10

🧠

BoostLoRA: Growing Effective Rank by Boosting Adapters

BoostLoRA introduces a gradient-boosting framework that enables parameter-efficient fine-tuning adapters to grow their effective rank iteratively, allowing ultra-low-parameter models to match or exceed full fine-tuning performance across mathematical reasoning, code generation, and protein classification tasks. The method merges adapters with zero inference overhead while maintaining minimal per-round parameter costs.

AIBullisharXiv – CS AI · Apr 106/10

🧠

LoRA-DA: Data-Aware Initialization for Low-Rank Adaptation via Asymptotic Analysis

Researchers introduce LoRA-DA, a new initialization method for Low-Rank Adaptation that leverages target-domain data and theoretical optimization principles to improve fine-tuning performance. The method outperforms existing initialization approaches across multiple benchmarks while maintaining computational efficiency.

AINeutralarXiv – CS AI · Mar 24/108

🧠

DirMixE: Harnessing Test Agnostic Long-tail Recognition with Hierarchical Label Vartiations

Researchers introduce DirMixE, a new machine learning approach for handling test-agnostic long-tail recognition problems where test data distributions are unknown and imbalanced. The method uses a hierarchical Mixture-of-Expert strategy with Dirichlet meta-distributions and includes a Latent Skill Finetuning framework for efficient parameter tuning of foundation models.