AIBullisharXiv – CS AI · Jun 97/10
🧠Researchers introduce an end-to-end framework for compressing Large Language Models through joint structural pruning and mixed-precision quantization that optimizes global error propagation rather than layer-wise errors. The approach demonstrates significant performance improvements at ultra-low bit precisions (1-3 bits), reducing perplexity by up to 21% compared to existing methods.
🏢 Perplexity
AIBullisharXiv – CS AI · Jun 87/10
🧠Researchers propose Neuron-Level Mixed-Precision Quantization Aware Training (NMP-QAT), a neural network compression technique that independently optimizes precision for individual neurons rather than entire layers. The method achieves better compression-accuracy trade-offs than existing approaches, making it particularly valuable for deploying AI models on resource-constrained edge devices in 6G networks.
AIBullisharXiv – CS AI · Jun 57/10
🧠Researchers introduce Channel-Wise Mixed-Precision Quantization (CMPQ), a novel technique that reduces Large Language Model memory requirements by assigning different precision levels to different weight channels based on activation patterns. The method enables fractional-bit quantization between 2-4 bits while preserving critical information through outlier extraction, addressing deployment constraints on edge devices.
AIBullisharXiv – CS AI · May 77/10
🧠EdgeRazor introduces a lightweight quantization framework that compresses large language models to 1.88-bit precision while maintaining performance superior to existing 3-bit methods. The approach combines mixed-precision quantization with knowledge distillation and achieves up to 15.1× faster decoding with 80% storage reduction, requiring significantly lower computational training budgets than comparable techniques.
AIBullisharXiv – CS AI · Apr 107/10
🧠Researchers propose an expert-wise mixed-precision quantization strategy for Mixture-of-Experts models that assigns bit-widths based on router gradient changes and neuron variance. The method achieves higher accuracy than existing approaches while reducing inference memory overhead on large-scale models like Switch Transformer and Mixtral with minimal computational overhead.
AINeutralarXiv – CS AI · Mar 56/10
🧠Researchers reproduced and analyzed severe accuracy degradation in BERT transformer models when applying post-training quantization, showing validation accuracy drops from 89.66% to 54.33%. The study found that structured activation outliers intensify with model depth, with mixed precision quantization being the most effective mitigation strategy.
AIBullisharXiv – CS AI · Feb 277/106
🧠Researchers developed a runtime-reconfigurable bitwise systolic array architecture for multi-precision quantized neural networks on FPGA hardware accelerators. The system achieves 1.3-3.6x speedup on mixed-precision models while supporting higher clock frequencies up to 250MHz, addressing the trade-off between hardware efficiency and inference accuracy.
AINeutralarXiv – CS AI · Jun 46/10
🧠Researchers introduce dMX, a differentiable mixed-precision quantization framework that enables dynamic floating-point bit-width assignment across different layers of large language models. The method uses continuous optimization with temperature-based annealing to efficiently compress models while maintaining accuracy, demonstrating improvements over existing quantization heuristics across multiple LLM families.
🏢 Perplexity🧠 Llama
AIBullisharXiv – CS AI · May 16/10
🧠Researchers present a mixed precision training framework for neural ODEs that reduces memory usage by ~50% and achieves up to 2x speedup while maintaining accuracy. The approach uses low-precision computations for velocity evaluations and intermediate states while preserving high precision for weights and gradient accumulation, addressing computational and memory bottlenecks in continuous-time neural network architectures.
AINeutralarXiv – CS AI · Apr 206/10
🧠Researchers have developed a precision-aware training time predictor for distributed deep learning that accounts for floating-point precision settings, achieving 9.8% prediction accuracy compared to 147.85% error in existing models that ignore precision variations. The work addresses a critical gap in resource allocation and cost estimation for AI training workloads, where precision choices can create 2.4x variations in training time.
AINeutralMarkTechPost · Apr 64/10
🧠A technical tutorial demonstrates implementing NVIDIA's Transformer Engine with mixed-precision acceleration, covering GPU setup, CUDA compatibility verification, and fallback execution handling. The guide focuses on practical deep learning workflow optimization using FP8 precision and benchmarking techniques.
🏢 Nvidia
