#latent-geometry News & Analysis

2 articles tagged with #latent-geometry. AI-curated summaries with sentiment analysis and key takeaways from 50+ sources.

2 articles

AINeutralarXiv – CS AI · Jun 46/10

🧠

The Loss Is Not Enough: Sampling Conditions and Inductive Bias in Contrastive Representation Learning

Researchers develop a theoretical framework proving that contrastive learning—a dominant self-supervised AI technique—requires specific sampling diversity conditions to recover meaningful latent geometry. They demonstrate that standard approaches can learn non-orthogonal representations and propose a corrected InfoNCE variant, with experiments showing that architectural inductive bias becomes critical when sampling diversity is limited.

AIBullisharXiv – CS AI · Mar 37/108

🧠

Maximizing the Spectral Energy Gain in Sub-1-Bit LLMs via Latent Geometry Alignment

Researchers introduce LittleBit-2, a new framework for extreme compression of large language models that achieves sub-1-bit quantization while maintaining performance comparable to 1-bit baselines. The method uses Internal Latent Rotation and Joint Iterative Quantization to solve geometric alignment issues in binary quantization, establishing new state-of-the-art results on Llama-2 and Llama-3 models.