#low-rank-decomposition News & Analysis

2 articles tagged with #low-rank-decomposition. AI-curated summaries with sentiment analysis and key takeaways from 50+ sources.

2 articles

AIBullisharXiv – CS AI · Jun 237/10

🧠

UniRank: Unified Rank Allocation for Low-Rank LLM Compression

Researchers propose UniRank, a new method for efficiently allocating ranks in low-rank decomposition of large language models by scoring components via local singular energy and global functional importance. The approach achieves up to 50% perplexity reduction compared to baseline methods without additional fine-tuning, addressing a key bottleneck in LLM compression.

🏢 Perplexity

AIBullisharXiv – CS AI · Apr 77/10

🧠

SoLA: Leveraging Soft Activation Sparsity and Low-Rank Decomposition for Large Language Model Compression

Researchers propose SoLA, a training-free compression method for large language models that combines soft activation sparsity and low-rank decomposition. The method achieves significant compression while improving performance, demonstrating 30% compression on LLaMA-2-70B with reduced perplexity from 6.95 to 4.44 and 10% better downstream task accuracy.

🏢 Perplexity