Bielik-Q2-Sharp: A Comparative Study of Extreme 2-bit Quantization Methods for a Polish 11B Language Model

Researchers successfully developed Bielik-Q2-Sharp, the first systematic evaluation of extreme 2-bit quantization for Polish language models, achieving near-baseline performance while significantly reducing model size. The study compared six quantization methods on an 11B parameter model, with the best variant maintaining 71.92% benchmark performance versus 72.07% baseline at just 3.26 GB.

• →First academic study of extreme 2-bit quantization applied to Polish large language models achieved near-baseline performance with dramatic size reduction.
• →QuIP# E8P12 method maintained 71.92% performance across 22 Polish benchmarks versus 72.07% baseline while reducing size to 3.26 GB.
• →QTIP achieved best per-bit efficiency at 79.4% accuracy with only 2.4 bits per weight and 3.27 GB model size.
• →Study revealed MC-generation dissociation where rotation-based methods preserve quality metrics but fail at text generation.
• →Entire research project completed by single researcher on cloud GPUs with just $285 budget, demonstrating cost-effective AI research approach.