#activation-outliers News & Analysis

4 articles tagged with #activation-outliers. AI-curated summaries with sentiment analysis and key takeaways from 50+ sources.

4 articles

AIBullisharXiv – CS AI · Jun 87/10

🧠

OffQ: Taming Structured Outliers in LLM Quantization by Offsetting

OffQ introduces a novel quantization technique for large language models that addresses activation outliers through an offsetting mechanism, enabling efficient W4A4KV4 low-bit quantization. The method uses top-1 PCA to identify outlier subspaces and concentrates high-magnitude activations into a single channel via rotation, then converts this into a shared offset to reduce standard deviation. This approach maintains uniform-grid quantization while improving accuracy across diverse LLM architectures.

AINeutralarXiv – CS AI · Mar 56/10

🧠

Activation Outliers in Transformer Quantization: Reproduction, Statistical Analysis, and Deployment Tradeoffs

Researchers reproduced and analyzed severe accuracy degradation in BERT transformer models when applying post-training quantization, showing validation accuracy drops from 89.66% to 54.33%. The study found that structured activation outliers intensify with model depth, with mixed precision quantization being the most effective mitigation strategy.

AINeutralarXiv – CS AI · Apr 146/10

🧠

ReSpinQuant: Efficient Layer-Wise LLM Quantization via Subspace Residual Rotation Approximation

ReSpinQuant introduces an efficient quantization framework for large language models that combines the expressivity of layer-wise adaptation with the computational efficiency of global rotation methods. By leveraging offline activation rotation fusion and residual subspace rotation matching, the approach achieves state-of-the-art performance on aggressive quantization schemes (W4A4, W3A3) without significant inference overhead.

AIBullisharXiv – CS AI · Apr 76/10

🧠

MUXQ: Mixed-to-Uniform Precision MatriX Quantization via Low-Rank Outlier Decomposition

Researchers propose MUXQ, a new quantization technique for large language models that addresses activation outliers through low-rank decomposition. The method enables efficient INT8 quantization while maintaining accuracy close to FP16, making it suitable for edge device deployment with NPU-based hardware.

🏢 Perplexity