y0news
AnalyticsDigestsSourcesTopicsRSSAICrypto

#parameter-dynamics News & Analysis

2 articles tagged with #parameter-dynamics. AI-curated summaries with sentiment analysis and key takeaways from 50+ sources.

2 articles
AINeutralarXiv – CS AI · 3d ago6/10
🧠

Why Steering Works: Toward a Unified View of Language Model Parameter Dynamics

Researchers present a unified framework for understanding how different methods control large language models—including fine-tuning, LoRA, and activation interventions—revealing a fundamental trade-off between steering strength and output quality. The analysis explains this through an activation manifold perspective and introduces SPLIT, a new steering method that improves control while better preserving model coherence.

AINeutralarXiv – CS AI · Mar 24/105
🧠

Optimizer-Induced Low-Dimensional Drift and Transverse Dynamics in Transformer Training

Researchers analyzed training trajectories in small transformer models, finding that parameter updates organize into a dominant drift direction with transverse dynamics. The study reveals that different optimizers (AdamW vs SGD) create substantially different trajectory geometries, with AdamW developing multi-dimensional structures while SGD produces more linear evolution.