#gradient-analysis News & Analysis

5 articles tagged with #gradient-analysis. AI-curated summaries with sentiment analysis and key takeaways from 50+ sources.

5 articles

AIBullisharXiv – CS AI · May 97/10

🧠

Rethinking Adapter Placement: A Dominant Adaptation Module Perspective

Researchers introduce DomLoRA, a parameter-efficient fine-tuning method that identifies a single 'dominant adaptation module' where most gradient energy concentrates, achieving superior performance with only 0.7% of standard LoRA's trainable parameters. The discovery reveals that optimal adapter placement is architecture-dependent but task-stable across instruction following, reasoning, and code generation applications.

AIBullisharXiv – CS AI · May 97/10

🧠

DeTrigger: A Gradient-Centric Approach to Backdoor Attack Mitigation in Federated Learning

DeTrigger is a new federated learning framework that uses gradient analysis to detect and neutralize backdoor attacks in distributed machine learning systems. The approach achieves 251x faster detection than existing methods while mitigating 98.9% of backdoor attacks with minimal accuracy loss, addressing a critical vulnerability in privacy-preserving collaborative AI training.

AIBullisharXiv – CS AI · Mar 57/10

🧠

Spectral Surgery: Training-Free Refinement of LoRA via Gradient-Guided Singular Value Reweighting

Researchers have developed Spectral Surgery, a training-free method to improve LoRA (Low-Rank Adaptation) model performance by reweighting singular values based on gradient sensitivity. The technique achieves significant performance gains (up to +4.4 points on CommonsenseQA) by adjusting only about 1,000 scalar coefficients without requiring retraining.

🧠 Llama

AINeutralarXiv – CS AI · May 276/10

🧠

The Strongest Teacher Is Not Always the Best Teacher: Student-Centric Answer Selection

Researchers demonstrate that the highest-performing teacher model doesn't necessarily provide the best training data for student models. They propose Student-Centric Answer Sampling (SCAS), a framework that selects answers based on their estimated learning value for specific students rather than teacher strength alone, showing consistent performance improvements across 30 teacher models and 8 tasks.

AINeutralarXiv – CS AI · Mar 176/10

🧠

Gradient Atoms: Unsupervised Discovery, Attribution and Steering of Model Behaviors via Sparse Decomposition of Training Gradients

Researchers introduce Gradient Atoms, an unsupervised method that decomposes AI model training gradients to discover interpretable behaviors without requiring predefined queries. The technique can identify model behaviors like refusal patterns and arithmetic capabilities, while also serving as effective steering vectors to control model outputs.