LiftQuant: Continuous Bit-Width LLM via Dimensional Lifting and Projection

Researchers introduce LiftQuant, a novel quantization framework enabling continuous bit-width control for Large Language Models by lifting weights into higher-dimensional space and projecting them back via 1-bit lattices. The approach bridges the gap between rigid integer bit-widths and real-world deployment constraints, allowing a 70B LLM to compress to 2.4 bits while maintaining hardware efficiency and outperforming existing 2-bit quantization methods.

LiftQuant addresses a fundamental limitation in current LLM deployment: quantization methods force models into discrete bit-widths (2-bit, 3-bit, etc.) that rarely align with actual hardware constraints and memory budgets. This creates inefficiency where models are over-compressed or under-utilized. The framework introduces mathematical sophistication through dimensional lifting—projecting low-dimensional weight vectors from a higher-dimensional space using simple 1-bit lattices. The effective bit-width becomes a flexible parameter determined by the ratio of lifted to original dimensions, enabling quasi-continuous control rather than discrete jumps.

The innovation carries practical significance for AI infrastructure. Existing quantization relies on complex codebooks or uniform quantizers that either lose expressive power or demand expensive decoding operations. LiftQuant captures Vector Quantization's expressive benefits while maintaining hardware friendliness through linear-only decoding paths and 1-bit uniform quantizers. This means deployment on edge devices and consumer GPUs becomes more efficient.

For developers and AI infrastructure companies, this represents meaningful progress toward optimal model compression. The demonstrated compression of a 70B model to 2.4 bits on 24GB GPUs opens deployment pathways previously unavailable at this performance level. The continuous control mechanism also enables dynamic adaptation—models can be adjusted in real-time based on available resources rather than requiring separate fine-tuned checkpoints for each target bit-width.

Future attention should focus on whether this approach scales to trillion-parameter models and whether the mathematical framework generalizes across different model architectures beyond LLMs. Adoption depends on community implementation and whether frameworks like vLLM integrate LiftQuant support.

• →LiftQuant enables continuous rather than discrete bit-width quantization through dimensional lifting and projection mechanisms.
• →A 70B LLM compressed to 2.4 bits outperforms existing state-of-the-art 2-bit quantized models on identical hardware.
• →The framework maintains hardware efficiency by relying solely on linear transformations and 1-bit quantizers during decoding.
• →Flexible bit-width control allows precise fitting of models to specific memory budgets instead of forcing rigid integer constraints.
• →The approach captures expressive power of Vector Quantization while improving computational efficiency for real-world deployment.