16 articles tagged with #parameter-efficient. AI-curated summaries with sentiment analysis and key takeaways from 50+ sources.
Researchers introduce Efficient Draft Adaptation (EDA), a framework that significantly reduces the cost of adapting draft models for speculative decoding when target LLMs are fine-tuned. EDA achieves superior performance through decoupled architecture, data regeneration, and smart sample selection while requiring substantially less training resources than full retraining.
Researchers propose Sequential Adaptive Steering (SAS), a new framework for controlling Large Language Model personalities at inference time without retraining. The method uses orthogonalized steering vectors to enable precise, multi-dimensional personality control by adjusting coefficients, validated on Big Five personality traits.
Researchers introduce DMTrack, a novel dual-adapter architecture for spatio-temporal multimodal tracking that achieves state-of-the-art performance with only 0.93M trainable parameters. The system uses two key modules - a spatio-temporal modality adapter and a progressive modality complementary adapter - to bridge gaps between different modalities and enable better cross-modality fusion.
Researchers introduce SVDecode, a new method for adapting large language models to specific tasks without extensive fine-tuning. The technique uses steering vectors during decoding to align output distributions with task requirements, improving accuracy by up to 5 percentage points while adding minimal computational overhead.
Researchers introduce NoRA (Non-linear Rank Adaptation), a new parameter-efficient fine-tuning method that overcomes the 'linear ceiling' limitations of traditional LoRA by using SiLU gating and structural dropout. NoRA achieves superior performance at rank 64 compared to LoRA at rank 512, demonstrating significant efficiency gains in complex reasoning tasks.
Researchers introduce VisPrompt, a framework that improves prompt learning for vision-language models by injecting visual semantic information to enhance robustness against label noise. The approach keeps pre-trained models frozen while adding minimal trainable parameters, demonstrating superior performance across seven benchmark datasets under both synthetic and real-world noisy conditions.
Researchers introduce SPARE, a new machine unlearning method for text-to-image diffusion models that efficiently removes unwanted concepts while preserving model performance. The two-stage approach uses parameter localization and self-distillation to achieve selective concept erasure with minimal computational overhead.
Researchers propose a new framework for improving safety in multimodal AI models by targeting unsafe relationships between objects rather than removing entire concepts. The approach uses parameter-efficient edits to suppress dangerous combinations while preserving benign uses of the same objects and relations.
Researchers introduce IGU-LoRA, a new parameter-efficient fine-tuning method for large language models that adaptively allocates ranks across layers using integrated gradients and uncertainty-aware scoring. The approach addresses limitations of existing methods like AdaLoRA by providing more stable and accurate layer importance estimates, consistently outperforming baselines across diverse tasks.
Researchers developed a new continual learning framework for human activity recognition (HAR) in IoT wearable devices that prevents AI models from forgetting previous tasks when learning new ones. The method uses gated adaptation to achieve 77.7% accuracy while reducing forgetting from 39.7% to 16.2%, training only 2% of parameters.
Researchers developed pMoE, a novel parameter-efficient fine-tuning method that combines multiple expert domains through specialized prompt tokens and dynamic dispatching. Testing across 47 visual adaptation tasks in classification and segmentation shows superior performance with improved computational efficiency compared to existing methods.
Researchers introduce NTK-CL, a new framework for parameter-efficient fine-tuning in continual learning that uses Neural Tangent Kernel theory to address catastrophic forgetting. The approach achieves state-of-the-art performance by tripling feature representation and implementing adaptive mechanisms to maintain task-specific knowledge while learning new tasks.
Researchers propose SERA, a new architecture for referring image segmentation that uses mixture-of-experts and expression-aware routing to improve pixel-level mask generation from natural language descriptions. The system introduces lightweight expert refinement stages and parameter-efficient tuning that updates less than 1% of backbone parameters while achieving superior performance on spatial localization and boundary delineation tasks.
Researchers trained a compact 1.5B parameter language model to solve beam physics problems using reinforcement learning with verifiable rewards, achieving 66.7% improvement in accuracy. However, the model learned pattern-matching templates rather than true physics reasoning, failing to generalize to topological changes despite mastering the same underlying equations.
Researchers propose TAP-SLF, a parameter-efficient framework for adapting Vision Foundation Models to multiple ultrasound medical imaging tasks simultaneously. The method uses task-aware prompting and selective layer fine-tuning to achieve effective performance while avoiding overfitting on limited medical data.
Researchers developed a cost-effective method to adapt large language models to minority dialects using continual pre-training and LoRA techniques, successfully improving Quebec French dialect performance with minimal computational resources. The study demonstrates that parameter-efficient fine-tuning can expand quality LLM access to underserved linguistic communities while updating only 1% of model parameters.
