#parameter-dynamics News & Analysis

3 articles tagged with #parameter-dynamics. AI-curated summaries with sentiment analysis and key takeaways from 50+ sources.

3 articles

AINeutralarXiv – CS AI · May 76/10

🧠

From Parameter Dynamics to Risk Scoring : Quantifying Sample-Level Safety Degradation in LLM Fine-tuning

Researchers have identified a critical vulnerability in LLM safety alignment where fine-tuning on benign samples causes parameters to drift toward unsafe behaviors, erasing safety gains from millions of preference examples. The study proposes SQSD, a method to quantify and score individual training samples by their contribution to safety degradation, with demonstrated transferability across different model architectures and scales.

AINeutralarXiv – CS AI · Apr 146/10

🧠

Why Steering Works: Toward a Unified View of Language Model Parameter Dynamics

Researchers present a unified framework for understanding how different methods control large language models—including fine-tuning, LoRA, and activation interventions—revealing a fundamental trade-off between steering strength and output quality. The analysis explains this through an activation manifold perspective and introduces SPLIT, a new steering method that improves control while better preserving model coherence.

AINeutralarXiv – CS AI · Mar 24/105

🧠

Optimizer-Induced Low-Dimensional Drift and Transverse Dynamics in Transformer Training

Researchers analyzed training trajectories in small transformer models, finding that parameter updates organize into a dominant drift direction with transverse dynamics. The study reveals that different optimizers (AdamW vs SGD) create substantially different trajectory geometries, with AdamW developing multi-dimensional structures while SGD produces more linear evolution.