STARFISH: faST Accuracy Recovery in pruned networks From Internal State Healing

Researchers introduce STARFISH, a novel neural network healing method that efficiently recovers accuracy lost during weight pruning by aligning pruned networks with original internal state representations using minimal unlabeled calibration data. The technique achieves up to 22% accuracy improvement over existing methods and recovers 82% of original performance after removing 75% of weights from vision transformers.

STARFISH addresses a critical bottleneck in deploying large neural networks at scale. Model pruning—removing redundant weights—enables faster inference essential for edge computing and real-time applications, but traditionally sacrifices accuracy. This new healing approach fundamentally changes the accuracy-efficiency tradeoff by using internal state alignment rather than traditional fine-tuning methods that require large labeled datasets.

The technique's efficiency stems from its core insight: pruned networks lose accuracy because their internal representations diverge from the original model, not necessarily due to missing parameters. By optimizing a pruned network to maintain alignment with the original network's hidden layer activations using only 0.4% of training data, STARFISH bypasses the expensive retraining process. This becomes particularly valuable for transformer-based architectures like ViT and DeiT, which dominate modern AI applications.

For the AI industry, this advancement directly impacts deployment economics. Practitioners can now prune models more aggressively without proportional accuracy losses, reducing computational costs and enabling deployment on resource-constrained devices. The method's reliance on unlabeled calibration data is especially significant in domains where labeled data remains expensive or proprietary.

Looking ahead, the implications extend across AI infrastructure providers, edge computing platforms, and enterprises deploying large language models. If these results generalize beyond vision transformers to other architectures, we may see widespread adoption in production systems. The next milestone involves testing STARFISH on large language models and understanding its performance boundaries under extreme pruning scenarios exceeding 90% weight removal.

• →STARFISH recovers 22% more accuracy than existing methods when removing 50% of network weights
• →The method requires only 0.4% of original training data as unlabeled calibration examples
• →Internal state alignment, not parameter count, drives the healing mechanism
• →Aggressive pruning of 75% weight removal retains 82% of original model accuracy on ImageNet
• →Results focus on vision transformers with unclear generalization to other architectures