56 articles tagged with #model-compression. AI-curated summaries with sentiment analysis and key takeaways from 50+ sources.
AINeutralarXiv – CS AI · 2d ago6/10
🧠ReSpinQuant introduces an efficient quantization framework for large language models that combines the expressivity of layer-wise adaptation with the computational efficiency of global rotation methods. By leveraging offline activation rotation fusion and residual subspace rotation matching, the approach achieves state-of-the-art performance on aggressive quantization schemes (W4A4, W3A3) without significant inference overhead.
AIBullisharXiv – CS AI · 3d ago6/10
🧠Researchers demonstrate that HiFloat4, a 4-bit floating-point format, enables efficient large language model training on Huawei's Ascend NPUs with up to 4x improvements in compute throughput and memory efficiency. The study shows that specialized stabilization techniques can maintain accuracy within 1% of full-precision baselines while preserving computational gains across dense and mixture-of-experts architectures.
AINeutralarXiv – CS AI · 3d ago6/10
🧠Researchers provide the first rigorous theoretical analysis of OPTQ (GPTQ), a widely-used post-training quantization algorithm for neural networks and LLMs, establishing quantitative error bounds and validating practical design choices. The study extends theoretical guarantees to both deterministic and stochastic variants of OPTQ and the Qronos algorithm, offering guidance for regularization parameter selection and quantization alphabet sizing.
AINeutralarXiv – CS AI · 3d ago6/10
🧠Researchers introduce CoA-LoRA, a method that dynamically adapts LoRA fine-tuning to different quantization configurations without requiring separate retraining for each setting. The approach uses a configuration-aware model and Pareto-based search to optimize low-rank adjustments across heterogeneous edge devices, achieving comparable performance to traditional methods with zero additional computational cost.
AIBullisharXiv – CS AI · Apr 76/10
🧠Researchers propose REAM (Router-weighted Expert Activation Merging), a new method for compressing large language models that groups and merges expert weights instead of pruning them. The technique preserves model performance better than existing pruning methods while reducing memory requirements for deployment.
AIBullisharXiv – CS AI · Apr 76/10
🧠Researchers developed a new method to train transformer neural networks using discrete cosine transform (DCT) coefficients, achieving the same performance while using only 52% of the parameters. The technique requires no architectural changes and simply replaces standard linear layers with spectral layers that store DCT coefficients instead of full weight matrices.
🏢 Perplexity
AIBullisharXiv – CS AI · Apr 76/10
🧠Researchers have developed DP-OPD (Differentially Private On-Policy Distillation), a new framework for training privacy-preserving language models that significantly improves performance over existing methods. The approach simplifies the training pipeline by eliminating the need for DP teacher training and offline synthetic text generation while maintaining strong privacy guarantees.
🏢 Perplexity
AIBullisharXiv – CS AI · Apr 66/10
🧠Researchers developed QAPruner, a new framework that simultaneously optimizes vision token pruning and post-training quantization for Multimodal Large Language Models (MLLMs). The method addresses the problem where traditional token pruning can discard important activation outliers needed for quantization stability, achieving 2.24% accuracy improvement over baselines while retaining only 12.5% of visual tokens.
AIBullisharXiv – CS AI · Mar 176/10
🧠Researchers introduce GPrune-LLM, a new structured pruning framework that improves compression of large language models by addressing calibration bias and cross-task generalization issues. The method partitions neurons into behavior-consistent modules and uses adaptive metrics based on distribution sensitivity, showing consistent improvements in post-compression performance.
AIBullisharXiv – CS AI · Mar 176/10
🧠Researchers developed a resource-efficient framework for compressing large language models using knowledge distillation and chain-of-thought reinforcement learning. The method successfully compressed Qwen 3B to 0.5B while retaining 70-95% of performance across English, Spanish, and coding tasks, making AI models more suitable for resource-constrained deployments.
AIBullisharXiv – CS AI · Mar 176/10
🧠Researchers developed SimCert, a probabilistic certification framework that verifies behavioral similarity between compressed neural networks and their original versions. The framework addresses critical safety challenges in deploying compressed DNNs on resource-constrained systems by providing quantitative safety guarantees with adjustable confidence levels.
AINeutralarXiv – CS AI · Mar 176/10
🧠Researchers conducted the first systematic study on post-training quantization for diffusion large language models (dLLMs), identifying activation outliers as a key challenge for compression. The study evaluated state-of-the-art quantization methods across multiple dimensions to provide insights for efficient dLLM deployment on edge devices.
AIBullisharXiv – CS AI · Mar 166/10
🧠Researchers introduce a new knowledge distillation framework that improves training of smaller AI models by using intermediate representations from large language models rather than their final outputs. The method shows consistent improvements across reasoning benchmarks, particularly when training data is limited, by providing cleaner supervision signals.
AIBullisharXiv – CS AI · Mar 126/10
🧠Researchers introduce HEAL (Hindsight Entropy-Assisted Learning), a new framework for distilling reasoning capabilities from large AI models into smaller ones. The method overcomes traditional limitations by using three core modules to bridge reasoning gaps and significantly outperforms standard distillation techniques.
🏢 Perplexity
AIBullisharXiv – CS AI · Mar 96/10
🧠Researchers introduce HiPP-Prune, a new framework for efficiently compressing vision-language models while maintaining performance and reducing hallucinations. The hierarchical approach uses preference-based pruning that considers multiple objectives including task utility, visual grounding, and compression efficiency.
AIBullisharXiv – CS AI · Mar 96/10
🧠Researchers introduced VLMQ, a post-training quantization framework specifically designed for vision-language models that addresses visual over-representation and modality gaps. The method achieves significant performance improvements, including 16.45% better results on MME-RealWorld under 2-bit quantization compared to existing approaches.
AIBullisharXiv – CS AI · Mar 37/107
🧠Researchers introduce Attn-QAT, the first systematic approach to 4-bit quantization-aware training for attention mechanisms in AI models. The method enables stable FP4 computation on emerging GPUs and delivers up to 1.5x speedup on RTX 5090 while maintaining model quality across diffusion and language models.
AIBullisharXiv – CS AI · Mar 37/108
🧠Researchers introduce LittleBit-2, a new framework for extreme compression of large language models that achieves sub-1-bit quantization while maintaining performance comparable to 1-bit baselines. The method uses Internal Latent Rotation and Joint Iterative Quantization to solve geometric alignment issues in binary quantization, establishing new state-of-the-art results on Llama-2 and Llama-3 models.
AIBullisharXiv – CS AI · Mar 36/107
🧠Researchers developed a new mathematical framework called Curvature-Weighted Capacity Allocation that optimizes large language model performance by identifying which layers contribute most to loss reduction. The method uses the Minimum Description Length principle to make principled decisions about layer pruning and capacity allocation under hardware constraints.
$NEAR
AIBullisharXiv – CS AI · Mar 36/103
🧠Researchers present a comprehensive analysis of post-training N:M activation pruning techniques for large language models, demonstrating that activation pruning preserves generative capabilities better than weight pruning. The study establishes hardware-friendly baselines and explores sparsity patterns beyond NVIDIA's standard 2:4, with 8:16 patterns showing superior performance while maintaining implementation feasibility.
AIBullisharXiv – CS AI · Mar 26/1017
🧠Researchers introduce Quant Experts (QE), a new post-training quantization technique for Vision-Language Models that uses adaptive error compensation with mixture-of-experts architecture. The method addresses computational and memory overhead issues by intelligently handling token-dependent and token-independent channels, maintaining performance comparable to full-precision models across 2B to 70B parameter scales.
AIBullisharXiv – CS AI · Feb 276/106
🧠Researchers introduce SideQuest, a novel KV cache management system that uses Large Reasoning Models to compress memory usage during long-horizon AI tasks. The system reduces peak token usage by up to 65% while maintaining accuracy by having the model itself determine which tokens are useful to keep in memory.
AIBullisharXiv – CS AI · Feb 276/106
🧠Researchers propose RL-aware distillation (RLAD), a new method to efficiently transfer knowledge from large language models to smaller ones during reinforcement learning training. The approach uses Trust Region Ratio Distillation (TRRD) to selectively guide student models only when it improves policy updates, outperforming existing distillation methods across reasoning benchmarks.
AIBullishHugging Face Blog · May 156/105
🧠Falcon-Edge represents a new series of 1.58-bit language models that are designed to be powerful, universal, and fine-tunable. These models appear to focus on efficiency through reduced bit precision while maintaining performance capabilities.
AIBullishHugging Face Blog · Apr 296/107
🧠Intel has introduced AutoRound, an advanced quantization technique designed to optimize Large Language Models (LLMs) and Vision-Language Models (VLMs). This technology aims to reduce model size and computational requirements while maintaining performance quality for AI applications.
