From Fewer Samples to Fewer Bits: Reframing Dataset Distillation as Joint Optimization of Precision and Compactness

Researchers propose QuADD (Quantization-aware Dataset Distillation), a new framework that jointly optimizes dataset compression and precision to create more efficient synthetic training datasets. The method integrates differentiable quantization within the distillation process, achieving better accuracy per bit than existing approaches on image classification and 3GPP beam management tasks.

• →QuADD introduces joint optimization of dataset compactness and precision under fixed bit budgets for more efficient training.
• →The framework integrates differentiable quantization directly into the distillation loop for end-to-end optimization.
• →Both uniform and adaptive non-uniform quantization are supported, with adaptive methods learning optimal quantization levels from data.
• →Experiments show QuADD outperforms existing dataset distillation and post-quantized baselines in accuracy per bit metrics.
• →The approach establishes a new standard for information-efficient dataset distillation across multiple tasks.