#low-bit-quantization News & Analysis

2 articles tagged with #low-bit-quantization. AI-curated summaries with sentiment analysis and key takeaways from 50+ sources.

2 articles

AIBullisharXiv – CS AI · Jun 87/10

🧠

OffQ: Taming Structured Outliers in LLM Quantization by Offsetting

OffQ introduces a novel quantization technique for large language models that addresses activation outliers through an offsetting mechanism, enabling efficient W4A4KV4 low-bit quantization. The method uses top-1 PCA to identify outlier subspaces and concentrates high-magnitude activations into a single channel via rotation, then converts this into a shared offset to reduce standard deviation. This approach maintains uniform-grid quantization while improving accuracy across diverse LLM architectures.

AIBullisharXiv – CS AI · Jun 27/10

🧠

BitsMoE: Efficient Spectral Energy-Guided Bit Allocation for MoE LLM Quantization

BitsMoE introduces a spectral-energy-guided quantization framework for compressing Mixture-of-Experts large language models, achieving significant improvements in the ultra-low-bit regime. The method uses SVD decomposition to intelligently allocate bits across expert weights, delivering 27.83 percentage point accuracy improvements over existing approaches at 2-bit quantization while accelerating inference speed by 1.76× on Qwen models.