AIBullisharXiv – CS AI · Jun 57/10
🧠Researchers introduce Channel-Wise Mixed-Precision Quantization (CMPQ), a novel technique that reduces Large Language Model memory requirements by assigning different precision levels to different weight channels based on activation patterns. The method enables fractional-bit quantization between 2-4 bits while preserving critical information through outlier extraction, addressing deployment constraints on edge devices.
AIBullisharXiv – CS AI · Jun 47/10
🧠Researchers introduce YAQA, a new quantization algorithm that improves model compression by directly optimizing end-to-end error rather than layer-by-layer error. The method achieves 30% error reduction compared to existing approaches like GPTQ and even outperforms quantization-aware training, with theoretical guarantees backing its performance.
AIBullisharXiv – CS AI · Jun 47/10
🧠Researchers systematically evaluate whether transformer models require three separate QKV projections, discovering that shared projection variants perform comparably while reducing computational overhead. The Q-K=V configuration achieves 50% KV cache reduction with minimal performance loss and combines effectively with existing optimization techniques like MQA to enable practical on-device deployment.
🏢 Perplexity
AIBullisharXiv – CS AI · Jun 47/10
🧠Researchers introduce LiftQuant, a novel quantization framework enabling continuous bit-width control for Large Language Models by lifting weights into higher-dimensional space and projecting them back via 1-bit lattices. The approach bridges the gap between rigid integer bit-widths and real-world deployment constraints, allowing a 70B LLM to compress to 2.4 bits while maintaining hardware efficiency and outperforming existing 2-bit quantization methods.
AIBullisharXiv – CS AI · Jun 47/10
🧠Researchers present Recover-LoRA, a technique that recovers accuracy in large language models aggressively quantized to 2-bit precision by applying low-rank adapters trained on synthetic data. The method achieves 7.5-23.3% throughput improvements while recovering 80-95% of lost accuracy on most benchmarks, enabling practical deployment of compressed models on edge devices.
AIBullisharXiv – CS AI · Jun 27/10
🧠Researchers introduce DOT-MoE, a framework that converts dense language models into sparse Mixture-of-Experts architectures using differentiable optimal transport. The method achieves 90% performance retention while reducing active parameters by 50%, addressing a critical bottleneck in LLM inference efficiency without the instability of training MoEs from scratch.
$DOT
AIBullisharXiv – CS AI · Jun 27/10
🧠Researchers propose LU-KV, a novel framework for optimizing KV cache eviction in large language models by formulating budget allocation as a combinatorial optimization problem. The approach reduces KV cache size by 80% while maintaining performance, significantly lowering inference latency and GPU memory requirements.
AIBullisharXiv – CS AI · Jun 27/10
🧠Researchers introduce STARFISH, a novel neural network healing method that efficiently recovers accuracy lost during weight pruning by aligning pruned networks with original internal state representations using minimal unlabeled calibration data. The technique achieves up to 22% accuracy improvement over existing methods and recovers 82% of original performance after removing 75% of weights from vision transformers.
AIBullisharXiv – CS AI · Jun 27/10
🧠Researchers propose ASKD-Whisper, a new knowledge distillation technique that compresses OpenAI's Whisper speech recognition model while improving performance. The method achieves 5x faster inference and 1.07% lower error rates than the original teacher model by dynamically reducing reliance on the teacher's predictions during training.
AIBullisharXiv – CS AI · Jun 17/10
🧠Researchers introduce MuCRASP, a structured pruning framework designed to compress vision-language models while preserving chain-of-thought reasoning capabilities. The method addresses limitations in existing pruning techniques by identifying reasoning-critical components and accounting for differences between visual and textual modalities, achieving superior performance preservation at 30-50% compression rates.
🏢 Perplexity
AIBullisharXiv – CS AI · Jun 17/10
🧠Researchers propose OBCache, a novel KV cache pruning framework that optimizes memory efficiency for long-context LLM inference by measuring token importance based on actual impact to attention outputs rather than heuristic attention weights. The method, grounded in Optimal Brain Damage theory, demonstrates consistent accuracy improvements over existing eviction strategies on LLaMA and Qwen models.
AIBullisharXiv – CS AI · May 297/10
🧠Researchers introduce Logit-aware Final-block Quantization (LFQ), a technique that improves low-bit quantization of large language models by optimizing the final transformer block to preserve token probability distributions. This advancement addresses quality degradation in generative tasks while maintaining efficiency gains critical for deploying scaled LLMs.
AIBullisharXiv – CS AI · May 297/10
🧠LoopFM introduces a novel knowledge distillation framework that transfers rich intermediate representations from large foundation models to compact vertical models, achieving significant conversion improvements (0.5-1.22%) in industrial-scale systems by structuring FM embeddings as input features rather than relying on single scalar predictions.
AIBullisharXiv – CS AI · May 297/10
🧠Researchers introduce BitTP, a quantization technique that compresses LLM-based trajectory prediction models to 1.58-bit weights while maintaining full-precision activations, enabling deployment on resource-constrained edge devices. The approach not only reduces memory and latency but actually improves prediction accuracy by 14-21% compared to full-precision baselines, demonstrating that strategic quantization can serve as an effective regularizer.
AIBullisharXiv – CS AI · May 297/10
🧠Researchers introduce OccamToken, a training-free method for compressing vision-language models by pruning unnecessary visual tokens while maintaining accuracy. The approach reduces visual token sequences by 98.6% (from 2,880 to 40 tokens) on LLaVA-NeXT while preserving over 93% accuracy, addressing computational bottlenecks in VLM inference.
AIBullisharXiv – CS AI · May 297/10
🧠Researchers introduce HARP, a learnable adaptive rotation processor that improves extreme low-bit quantization for large language models by replacing fixed Hadamard transforms with optimizable structured orthogonal processors. The technique maintains full-precision equivalence while achieving better perplexity and accuracy across 2-4 bit quantization settings on models up to 70B parameters, with deployment speeds competitive with standard approaches.
🏢 Perplexity
AIBullisharXiv – CS AI · May 297/10
🧠Researchers present PeRQ, a post-training quantization method that uses permutations to optimize block rotations for neural network compression. The approach recovers up to 90% of full-vector rotation performance when quantizing large language models to INT4, significantly outperforming existing block rotation methods.
🏢 Perplexity🧠 Llama
AIBullisharXiv – CS AI · May 297/10
🧠Researchers have developed a method to improve how large language models verify factual claims by framing fact-checking as a true/false reading comprehension task with explicit test-taking strategies. The approach reduces token usage by over 80% while maintaining competitive performance, and enables smaller language models to perform similarly to larger ones through fine-tuning and self-revision mechanisms.
AIBullisharXiv – CS AI · May 287/10
🧠Researchers propose Group-Query Latent Attention (GQLA), an advancement of DeepSeek's Multi-head Latent Attention that enables hardware-adaptive decoding through two algebraically equivalent inference paths without requiring model retraining. The innovation allows a single trained model to optimize performance across different hardware platforms—H100 GPUs and export-restricted H20 chips—while maintaining computational efficiency and supporting distributed tensor parallelism.
AIBullisharXiv – CS AI · May 287/10
🧠Researchers propose LIFT and PLACE, a knowledge distillation framework that enables stable training of extremely lightweight diffusion models by decomposing the teacher's complex denoising process into coarse and fine stages with spatially adaptive guidance. The method achieves stable convergence even at extreme compression ratios (1.6% of teacher size) where conventional distillation fails, with potential applications across image generation, latent diffusion, and flow-based models.
AIBullisharXiv – CS AI · May 287/10
🧠Researchers propose Hurwitz Quaternion Multiplicative Quantization (HQMQ), a calibration-free method for compressing KV caches in large language models using quaternion mathematics. The technique achieves 5x compression with minimal perplexity loss, matching full-precision performance at ~5 bits while outperforming existing quantization methods across five major model architectures.
🧠 Llama
AIBullisharXiv – CS AI · May 287/10
🧠Google researchers unveiled BlazeEdit, a 195M-parameter image-to-image diffusion model optimized for on-device mobile deployment, eliminating text-conditioning to handle object removal, outpainting, tone correction, relighting, and sticker generation. The model completes inference in 290ms on Pixel 10 while maintaining competitive quality, advancing the trend toward privacy-preserving edge AI.
AIBullisharXiv – CS AI · May 287/10
🧠Researchers introduce TSVD, a framework for training Large Language Models more efficiently by maintaining low-rank representations and strict weight orthonormality throughout pretraining. The method uses adaptive rank selection and caching mechanisms to reduce computational overhead while matching or exceeding the performance of standard full-parameter models.
AIBullisharXiv – CS AI · May 287/10
🧠Researchers present a framework for converting Mixture-of-Experts (MoE) language models into standard dense architectures through expert selection, grouping, and knowledge distillation. The method achieves superior performance compared to traditional dense-to-dense pruning while enabling deployment on memory-constrained systems.
AIBullisharXiv – CS AI · May 277/10
🧠Researchers develop a systematic approach to quantization-aware training for large language models using 8-bit floating-point formats, identifying and solving two critical failure modes—amax saturation and catastrophic forgetting—that don't surface in standard training metrics. Their solution achieves near-lossless performance with only 0.43% degradation on benchmark tasks, advancing practical LLM deployment efficiency.
