AIBullisharXiv – CS AI · 3d ago7/10
🧠Researchers propose improved post-training quantization techniques for large language models using quantile-robust scaling policies and learned channel scales, demonstrating 18.5% error reduction on LLaMA-3.2-1B under W4A4 quantization. The work addresses activation quantization challenges caused by outlier-dominated channels, offering practical efficiency improvements for LLM deployment without requiring full model retraining.
AIBullisharXiv – CS AI · 3d ago7/10
🧠Researchers introduce PiSO (Piecewise Scale Optimization), an algorithm that optimizes quantization scaling factors for compressing large language models more effectively than existing heuristic methods. By using calibration data to compute optimal channel-wise scales, PiSO demonstrates consistent improvements in model perplexity and downstream accuracy across Llama and Qwen models, with gains becoming more pronounced at lower bit-widths.
🏢 Perplexity🧠 Llama
AIBullisharXiv – CS AI · 4d ago7/10
🧠ScaleSweep introduces an optimized block scale initialization method for NVFP4 quantization of large language models, improving upon traditional AbsMax approaches. The technique theoretically bounds the search space and empirically achieves 93% performance retention under aggressive 4-bit quantization, advancing hardware-efficient AI inference.
🧠 Llama
AIBullisharXiv – CS AI · Jun 57/10
🧠SAGE-PTQ introduces a novel ultra-low-bit quantization framework for large language models that dramatically reduces scaling overhead while maintaining accuracy. The method achieves 1.03 weight bits per parameter with minimal scaling costs, outperforming existing approaches like BiLLM by orders of magnitude in perplexity metrics while requiring significantly less GPU memory.
🏢 Nvidia🏢 Perplexity
AIBullisharXiv – CS AI · Jun 47/10
🧠QuBLAST is a new post-training quantization method that compresses large language models by 40-45% while maintaining performance, using block-level mixed-precision quantization and activation scaling to address computational and memory constraints in LLM deployment.
🏢 Perplexity🧠 Llama
AIBullisharXiv – CS AI · May 297/10
🧠Researchers introduce Logit-aware Final-block Quantization (LFQ), a technique that improves low-bit quantization of large language models by optimizing the final transformer block to preserve token probability distributions. This advancement addresses quality degradation in generative tasks while maintaining efficiency gains critical for deploying scaled LLMs.
AIBullisharXiv – CS AI · May 297/10
🧠Researchers present PeRQ, a post-training quantization method that uses permutations to optimize block rotations for neural network compression. The approach recovers up to 90% of full-vector rotation performance when quantizing large language models to INT4, significantly outperforming existing block rotation methods.
🏢 Perplexity🧠 Llama
AIBullisharXiv – CS AI · May 277/10
🧠Researchers introduce InfoQuant, a training-free method that optimizes activation distributions for low-bit quantization in large language models by using Peak Suppression Orthogonal Transformation. The technique achieves 97% accuracy preservation under W4A4KV4 quantization and reduces performance degradation by 42% compared to previous methods, advancing efficient LLM deployment.
AIBullisharXiv – CS AI · May 97/10
🧠Researchers propose SARQC, a new post-training quantization framework for large language models that adds saliency-aware regularization to prevent quantized weights from drifting too far from original values. The method improves generalization performance across dense and mixture-of-experts LLMs without increasing inference costs.
🏢 Perplexity
AIBullisharXiv – CS AI · May 47/10
🧠Researchers introduce BWLA, a post-training quantization framework that achieves 1-bit weight compression alongside low-bit activations for large language models, addressing a critical bottleneck in LLM deployment. The method delivers 3.26× inference speedup on Qwen3-32B while maintaining competitive accuracy, potentially enabling more efficient LLM inference across resource-constrained environments.
🏢 Perplexity
AIBullisharXiv – CS AI · Apr 107/10
🧠Researchers introduce MoBiE, a novel binarization framework designed specifically for Mixture-of-Experts large language models that achieves significant efficiency gains through weight compression while maintaining model performance. The method addresses unique challenges in quantizing MoE architectures and demonstrates over 2× inference speedup with substantial perplexity reductions on benchmark models.
🏢 Perplexity
AINeutralarXiv – CS AI · 5d ago6/10
🧠Researchers propose FAIR-Calib, a novel post-training quantization framework designed to address instability issues in Diffusion Large Language Models (dLLMs) where early token decisions become permanently locked despite remaining fragile. The two-stage method uses frontier-aware reweighting to protect critical decision points during model compression, demonstrating improved performance over existing quantization baselines.
🏢 Meta
AINeutralarXiv – CS AI · May 276/10
🧠Researchers have developed Tail-Aware HiFloat4, a post-training quantization method that compresses text-to-video generation models using W4A4 (4-bit weights and activations) while maintaining output quality. The technique introduces activation-tail-aware calibration to handle statistical outliers, enabling efficient model deployment without retraining.
AINeutralarXiv – CS AI · Apr 146/10
🧠ReSpinQuant introduces an efficient quantization framework for large language models that combines the expressivity of layer-wise adaptation with the computational efficiency of global rotation methods. By leveraging offline activation rotation fusion and residual subspace rotation matching, the approach achieves state-of-the-art performance on aggressive quantization schemes (W4A4, W3A3) without significant inference overhead.
AINeutralarXiv – CS AI · Apr 136/10
🧠Researchers provide the first rigorous theoretical analysis of OPTQ (GPTQ), a widely-used post-training quantization algorithm for neural networks and LLMs, establishing quantitative error bounds and validating practical design choices. The study extends theoretical guarantees to both deterministic and stochastic variants of OPTQ and the Qronos algorithm, offering guidance for regularization parameter selection and quantization alphabet sizing.
AINeutralarXiv – CS AI · Mar 176/10
🧠Researchers conducted the first systematic study on post-training quantization for diffusion large language models (dLLMs), identifying activation outliers as a key challenge for compression. The study evaluated state-of-the-art quantization methods across multiple dimensions to provide insights for efficient dLLM deployment on edge devices.
AIBullisharXiv – CS AI · Mar 26/1017
🧠Researchers introduce Quant Experts (QE), a new post-training quantization technique for Vision-Language Models that uses adaptive error compensation with mixture-of-experts architecture. The method addresses computational and memory overhead issues by intelligently handling token-dependent and token-independent channels, maintaining performance comparable to full-precision models across 2B to 70B parameter scales.
