#vector-quantization News & Analysis

7 articles tagged with #vector-quantization. AI-curated summaries with sentiment analysis and key takeaways from 50+ sources.

7 articles

AIBullisharXiv – CS AI · Jun 107/10

🧠

LC-QAT: Data-Efficient 2-Bit QAT for LLMs via Linear-Constrained Vector Quantization

Researchers introduce LC-QAT, a novel 2-bit quantization method for large language models that combines vector quantization with learnable affine mappings to achieve superior compression with minimal training data. The approach outperforms existing quantization-aware training methods while requiring only 0.1-10% of typical training data, advancing the practical deployment of extremely low-bit LLMs.

AIBullisharXiv – CS AI · Jun 47/10

🧠

LiftQuant: Continuous Bit-Width LLM via Dimensional Lifting and Projection

Researchers introduce LiftQuant, a novel quantization framework enabling continuous bit-width control for Large Language Models by lifting weights into higher-dimensional space and projecting them back via 1-bit lattices. The approach bridges the gap between rigid integer bit-widths and real-world deployment constraints, allowing a 70B LLM to compress to 2.4 bits while maintaining hardware efficiency and outperforming existing 2-bit quantization methods.

AIBullisharXiv – CS AI · Jun 236/10

🧠

VQ4SNN: Vector Quantization for Memory-Efficient FPGA Spiking Neural Networks

Researchers propose VQ4SNN, a hardware-efficient architecture that uses vector quantization to reduce memory requirements for spiking neural networks on FPGAs by 52-61% without sacrificing inference accuracy. This innovation addresses a critical bottleneck in deploying dense SNNs on edge hardware, combining weight-sharing techniques with FPGA-aware memory optimization.

AINeutralarXiv – CS AI · Jun 116/10

🧠

Cross-Layer Discrete Concept Discovery for Interpreting Language Models

Researchers introduce CLVQ-VAE, a novel framework for interpreting language models by discovering discrete, interpretable concepts across layers. The method outperforms existing approaches by collapsing duplicated features in residual streams into compact concept vectors, achieving 93% accuracy drops when concepts are removed and 78% human prediction recovery from visualizations.

AINeutralarXiv – CS AI · Jun 26/10

🧠

Graph is a Natural Regularization: Revisiting Vector Quantization for Graph Representation Learning

Researchers propose RGVQ, a novel framework addressing codebook collapse in Vector Quantization for graph neural networks, a technical limitation that degrades token expressiveness and generalization. By integrating graph topology as regularization and introducing soft assignments, RGVQ improves codebook utilization across downstream graph learning tasks.

AINeutralarXiv – CS AI · Jun 26/10

🧠

Channel-wise Vector Quantization

Researchers introduce Channel-wise Vector Quantization (CVQ), a novel image tokenization method that quantizes individual channels rather than spatial patches, paired with a Channel-wise Autoregressive (CAR) generation model that produces images by progressively refining visual details. The approach achieves 100% codebook utilization and demonstrates strong performance on text-to-image generation benchmarks, suggesting a fundamentally different approach to visual AI tasks.

AINeutralarXiv – CS AI · May 126/10

🧠

Fitting Multilinear Polynomials for Logic Gate Networks

Researchers propose a novel approach to training learnable logic gate networks by representing 2-input Boolean gates as multilinear polynomials in 4-dimensional space, reducing a vector-quantization problem from 16 to 4 parameters per neuron. The CovJac method outperforms the baseline Soft-Mix approach, particularly at network depth, by addressing gradient starvation issues that cause performance collapse in deeper architectures.