AngelSlim: A more accessible, comprehensive, and efficient toolkit for large model compression

Tencent Hunyuan team introduces AngelSlim, a comprehensive toolkit for large model compression featuring quantization, speculative decoding, and pruning techniques. The toolkit includes the first industrially viable 2-bit large model (HY-1.8B-int2) and achieves 1.8x to 2.0x throughput gains while maintaining output quality.

• →AngelSlim consolidates cutting-edge compression algorithms including quantization, speculative decoding, token pruning, and distillation into a unified pipeline.
• →The toolkit features HY-1.8B-int2 as the first industrially viable 2-bit large model, pushing ultra-low-bit quantization boundaries.
• →Training-aligned speculative decoding framework achieves 1.8x to 2.0x throughput improvements without compromising output correctness.
• →Training-free sparse attention framework reduces Time-to-First-Token in long-context scenarios through hybrid static and dynamic token selection.
• →Specialized pruning strategies for multimodal models include IDPruner for vision tokens and Samp for adaptive audio token processing.