117 articles tagged with #model-compression. AI-curated summaries with sentiment analysis and key takeaways from 50+ sources.
Researchers introduce HiPP-Prune, a new framework for efficiently compressing vision-language models while maintaining performance and reducing hallucinations. The hierarchical approach uses preference-based pruning that considers multiple objectives including task utility, visual grounding, and compression efficiency.
Researchers introduced VLMQ, a post-training quantization framework specifically designed for vision-language models that addresses visual over-representation and modality gaps. The method achieves significant performance improvements, including 16.45% better results on MME-RealWorld under 2-bit quantization compared to existing approaches.
Researchers introduce Attn-QAT, the first systematic approach to 4-bit quantization-aware training for attention mechanisms in AI models. The method enables stable FP4 computation on emerging GPUs and delivers up to 1.5x speedup on RTX 5090 while maintaining model quality across diffusion and language models.
Researchers introduce LittleBit-2, a new framework for extreme compression of large language models that achieves sub-1-bit quantization while maintaining performance comparable to 1-bit baselines. The method uses Internal Latent Rotation and Joint Iterative Quantization to solve geometric alignment issues in binary quantization, establishing new state-of-the-art results on Llama-2 and Llama-3 models.
Researchers developed a new mathematical framework called Curvature-Weighted Capacity Allocation that optimizes large language model performance by identifying which layers contribute most to loss reduction. The method uses the Minimum Description Length principle to make principled decisions about layer pruning and capacity allocation under hardware constraints.
Researchers present a comprehensive analysis of post-training N:M activation pruning techniques for large language models, demonstrating that activation pruning preserves generative capabilities better than weight pruning. The study establishes hardware-friendly baselines and explores sparsity patterns beyond NVIDIA's standard 2:4, with 8:16 patterns showing superior performance while maintaining implementation feasibility.
Researchers introduce Quant Experts (QE), a new post-training quantization technique for Vision-Language Models that uses adaptive error compensation with mixture-of-experts architecture. The method addresses computational and memory overhead issues by intelligently handling token-dependent and token-independent channels, maintaining performance comparable to full-precision models across 2B to 70B parameter scales.
Researchers introduce SideQuest, a novel KV cache management system that uses Large Reasoning Models to compress memory usage during long-horizon AI tasks. The system reduces peak token usage by up to 65% while maintaining accuracy by having the model itself determine which tokens are useful to keep in memory.
Researchers propose RL-aware distillation (RLAD), a new method to efficiently transfer knowledge from large language models to smaller ones during reinforcement learning training. The approach uses Trust Region Ratio Distillation (TRRD) to selectively guide student models only when it improves policy updates, outperforming existing distillation methods across reasoning benchmarks.
Falcon-Edge represents a new series of 1.58-bit language models that are designed to be powerful, universal, and fine-tunable. These models appear to focus on efficiency through reduced bit precision while maintaining performance capabilities.
Intel has introduced AutoRound, an advanced quantization technique designed to optimize Large Language Models (LLMs) and Vision-Language Models (VLMs). This technology aims to reduce model size and computational requirements while maintaining performance quality for AI applications.
The article discusses AutoGPTQ, a technique for making large language models more efficient and lightweight through quantization. This approach reduces model size and computational requirements while maintaining performance, making AI models more accessible for deployment.
Stability AI has open-sourced knowledge distillation code and model weights for SD-Small and SD-Tiny, making smaller and more efficient versions of Stable Diffusion available to the community. This release enables developers to run image generation models with reduced computational requirements while maintaining reasonable quality.
The article discusses block sparse matrices as a technique to create smaller and faster language models. This approach could significantly reduce computational requirements and memory usage in AI systems while maintaining performance.
Researchers propose Text-guided Multi-view Knowledge Distillation (TMKD), a new method that uses dual-modality teachers (visual and text) to improve knowledge transfer from large AI models to smaller ones. The approach enhances visual teachers with multi-view inputs and incorporates CLIP text guidance, achieving up to 4.49% performance improvements across five benchmarks.
The article appears to discuss quantization backends in Diffusers, a machine learning library for diffusion models. However, the article body is empty, preventing detailed analysis of the technical content or implications.
The article appears to have an empty body, containing only a title about quantization schemes in Hugging Face Transformers. Without article content, this represents an incomplete or improperly loaded technical documentation piece about AI model optimization techniques.
