#task-selection News & Analysis

Learning What Matters: Probabilistic Task Selection via Mutual Information for Model Finetuning

Researchers introduce TaskPGM, a framework that optimizes how training data is distributed across multiple tasks when fine-tuning large language models by modeling task relationships through an energy-based probabilistic approach. The method balances task coverage against redundancy, demonstrating improvements over conventional uniform or size-proportional sampling strategies across multiple model families and evaluation benchmarks.