Maximizing the Spectral Energy Gain in Sub-1-Bit LLMs via Latent Geometry Alignment

Researchers introduce LittleBit-2, a new framework for extreme compression of large language models that achieves sub-1-bit quantization while maintaining performance comparable to 1-bit baselines. The method uses Internal Latent Rotation and Joint Iterative Quantization to solve geometric alignment issues in binary quantization, establishing new state-of-the-art results on Llama-2 and Llama-3 models.

• →LittleBit-2 achieves new state-of-the-art performance in sub-1-bit model compression (0.1-1 bpp) for large language models.
• →The framework solves latent geometry misalignment issues that previously prevented binary quantization from reaching its theoretical potential.
• →Internal Latent Rotation and Joint Iterative Quantization enable extreme compression with zero inference overhead.
• →Results on Llama-2 and Llama-3 demonstrate performance matching leading 1-bit methods while using significantly less memory.
• →The research identifies spectral energy gain as a key factor in successful extreme model compression for heavy-tailed spectra.