#llm-pruning News & Analysis

2 articles tagged with #llm-pruning. AI-curated summaries with sentiment analysis and key takeaways from 50+ sources.

2 articles

AIBullisharXiv – CS AI · Jun 96/10

🧠

LEAP: Learnable End-to-End Adaptive Pruning of Large Language Models

Researchers introduce LEAP, a new technique for pruning large language models that uses learnable per-weight masks to achieve better accuracy than existing layer-wise methods, particularly at aggressive sparsity levels. The approach replaces earlier intractable parameterization methods with a Bernoulli-via-Gumbel-sigmoid relaxation, demonstrating 2.59 points average improvement over ADMM across multiple LLM families.

AIBearisharXiv – CS AI · May 16/10

🧠

Junk DNA Hypothesis: Pruning Small Pre-Trained Weights Irreversibly and Monotonically Impairs "Difficult" Downstream Tasks in LLMs

Researchers challenge the conventional wisdom that large language models contain significant redundant parameters, demonstrating that small-magnitude weights encode crucial knowledge for difficult downstream tasks. The study reveals that pruning these weights causes irreversible performance degradation that cannot be recovered through continued training, with effects monotonically correlated to task difficulty.