HASA: Subnet Allocation for Compute-Constrained Model-Heterogeneous Federated Learning

Researchers propose HASA, a subnet allocation algorithm for federated learning that assigns model sizes to edge devices based on data heterogeneity rather than just compute constraints. The method improves prediction accuracy across distributed clients while maintaining fixed computational budgets, with implications for efficient on-device AI deployment.

HASA addresses a fundamental challenge in federated learning: how to optimally distribute computational resources across heterogeneous clients when both device capabilities and data distributions vary significantly. Traditional subnet allocation policies prioritize device constraints, treating computational budget as the primary optimization variable. This paper demonstrates that explicitly accounting for statistical heterogeneity—the variation in local data distributions across clients—yields measurable performance improvements without additional resource consumption.

The research builds on growing recognition that federated learning deployments face dual heterogeneity problems. Edge devices have vastly different processing capabilities, storage, and network connectivity. Simultaneously, clients generate data with different statistical properties, creating optimization challenges that compute-only allocation strategies cannot address. HASA's contribution lies in its simplicity: a training-phase rule that scores clients' heterogeneity from their local data and assigns smaller subnets to homogeneous clients while allocating larger model capacity to more heterogeneous ones.

Experimental validation shows meaningful but modest improvements—mean accuracy gains of 0.5 percentage points on next-word prediction tasks—with more pronounced benefits in worst-case client performance. The cross-domain image classification study reveals important limitations: heterogeneity-awareness only improves outcomes when the scoring metric accurately reflects clients' actual model capacity needs. This suggests the approach's effectiveness depends on domain-specific tuning rather than universal applicability.

For the federated learning ecosystem, HASA represents incremental optimization progress rather than fundamental innovation. The work validates that statistical properties merit consideration alongside resource constraints, informing future resource allocation frameworks. Success depends on practitioners accurately measuring heterogeneity metrics relevant to their specific applications.

• →HASA improves federated learning accuracy by allocating subnet sizes based on data heterogeneity rather than compute constraints alone.
• →Mean client accuracy improved 0.5 percentage points on benchmark tasks while worst-client performance gains were more pronounced.
• →Heterogeneity-aware allocation requires domain-specific heterogeneity scoring metrics to be effective across different applications.
• →The method maintains fixed computational budgets, enabling fair performance comparisons with alternative allocation strategies.
• →Assigning smaller models to heterogeneous clients degrades performance, confirming the opposite allocation strategy is optimal.