AIBullisharXiv – CS AI · 5d ago7/10
🧠Researchers demonstrate that parameter-efficient fine-tuning (PEFT) methods like adapters and LoRA can achieve competitive performance on instance segmentation tasks while training only 1-6% of model parameters, compared to 40-55% in traditional fine-tuning. The findings highlight that context-specific optimization is crucial, with 2-3 adapters per transformer block providing optimal efficiency gains.
AIBullisharXiv – CS AI · May 127/10
🧠Echo-LoRA introduces a parameter-efficient fine-tuning method that injects cross-layer representations from deeper neural network layers into shallow LoRA modules during training, achieving 3-5.7% performance improvements on reasoning tasks without adding inference costs. The technique discards its auxiliary training path post-deployment, maintaining the efficiency benefits of standard LoRA while delivering measurable capability gains.
AIBullisharXiv – CS AI · May 97/10
🧠Researchers introduce DomLoRA, a parameter-efficient fine-tuning method that identifies a single 'dominant adaptation module' where most gradient energy concentrates, achieving superior performance with only 0.7% of standard LoRA's trainable parameters. The discovery reveals that optimal adapter placement is architecture-dependent but task-stable across instruction following, reasoning, and code generation applications.
AIBullisharXiv – CS AI · May 296/10
🧠Researchers introduce NaRA (Noise-aware Low-Rank Adaptation), a parameter-efficient fine-tuning method designed specifically for diffusion large language models that adapts to noise levels during the denoising process. Unlike existing methods like LoRA that use static parameters, NaRA employs a hypernetwork to dynamically adjust low-rank matrices based on noise, achieving better performance on reasoning and code generation tasks.
AIBullisharXiv – CS AI · May 126/10
🧠Researchers introduce CERSA, a novel parameter-efficient fine-tuning method that uses singular value decomposition to reduce memory consumption while fine-tuning large language models. The technique outperforms existing methods like LoRA by capturing more rank characteristics of weight modifications while requiring substantially less memory for frozen weights.
AINeutralarXiv – CS AI · May 126/10
🧠AdaPreLoRA addresses a fundamental challenge in fine-tuning large language models by proposing a new optimization method that combines Adafactor preconditioning with Low-Rank Adaptation. The technique achieves competitive or superior performance across multiple benchmarks while maintaining memory efficiency comparable to standard LoRA optimizers.
AIBullisharXiv – CS AI · Apr 106/10
🧠Researchers propose FLeX, a parameter-efficient fine-tuning approach combining LoRA, advanced optimizers, and Fourier-based regularization to enable cross-lingual code generation across programming languages. The method achieves 42.1% pass@1 on Java tasks compared to a 34.2% baseline, demonstrating significant improvements in multilingual transfer without full model retraining.
🧠 Llama
AIBullisharXiv – CS AI · Mar 126/10
🧠Researchers conducted the first comprehensive evaluation of parameter-efficient fine-tuning (PEFT) for multi-task code analysis, showing that a single PEFT module can match full fine-tuning performance while reducing computational costs by up to 85%. The study found that even 1B-parameter models with multi-task PEFT outperform large general-purpose LLMs like DeepSeek and CodeLlama on code analysis tasks.
AIBullisharXiv – CS AI · Mar 36/104
🧠TiTok is a new framework for transferring LoRA (Low-Rank Adaptation) parameters between different Large Language Model backbones without requiring additional training data or discriminator models. The method uses token-level contrastive learning to achieve 4-10% performance gains over existing approaches in parameter-efficient fine-tuning scenarios.
AIBullisharXiv – CS AI · Feb 276/104
🧠Researchers propose an agentic AI framework using multiple LLM-based agents to optimize cell-free Open RAN networks through intent-driven automation. The system reduces active radio units by 42% in energy-saving mode while cutting memory usage by 92% through parameter-efficient fine-tuning.
AINeutralarXiv – CS AI · Mar 54/10
🧠Researchers introduce DevRev-Search, an automated system for building multi-tenant search systems that addresses the challenge of underutilized query data in enterprise environments. The system uses LLM-based relevance labeling and query-only fine-tuning to improve search performance while avoiding costly full corpus re-indexing.
