#gradient-based-methods News & Analysis

2 articles tagged with #gradient-based-methods. AI-curated summaries with sentiment analysis and key takeaways from 50+ sources.

2 articles

AINeutralarXiv – CS AI · Jun 26/10

🧠

AlphaToken: Decoupling Adaptation and Stability for Path-Aware Response Token Valuation in LLM Post-Training

Researchers introduce AlphaToken, a framework that improves large language model post-training by valuating individual response tokens based on their contribution to both task adaptation and preservation of pre-trained knowledge. The method uses gradient-based signals and a Fisher-drift proxy to identify high-value tokens, enabling more efficient fine-tuning and preference optimization while reducing catastrophic forgetting.

AINeutralarXiv – CS AI · May 276/10

🧠

Faithfulness Evaluation for Decoder-only LLM Attributions with Controlled Retained Information

Researchers propose π-Soft-NC and π-Soft-NS, improved evaluation metrics for assessing input attribution methods in large language models that control for the number of retained words, addressing a fundamental bias in existing faithfulness evaluation frameworks. They also introduce Grad-ELLM, a gradient-based attribution method designed for decoder-only LLMs that combines gradient and attention mechanisms for stronger explanatory performance.

🧠 Llama