#mixture-of-experts2 articles

2 articles

AIBullisharXiv – CS AI · 4h ago9

🧠

Quant Experts: Token-aware Adaptive Error Reconstruction with Mixture of Experts for Large Vision-Language Models Quantization

Researchers introduce Quant Experts (QE), a new post-training quantization technique for Vision-Language Models that uses adaptive error compensation with mixture-of-experts architecture. The method addresses computational and memory overhead issues by intelligently handling token-dependent and token-independent channels, maintaining performance comparable to full-precision models across 2B to 70B parameter scales.

AINeutralarXiv – CS AI · 4h ago1

🧠

DirMixE: Harnessing Test Agnostic Long-tail Recognition with Hierarchical Label Vartiations

Researchers introduce DirMixE, a new machine learning approach for handling test-agnostic long-tail recognition problems where test data distributions are unknown and imbalanced. The method uses a hierarchical Mixture-of-Expert strategy with Dirichlet meta-distributions and includes a Latent Skill Finetuning framework for efficient parameter tuning of foundation models.