AIBearisharXiv – CS AI · Apr 137/10
🧠
On the Limits of Layer Pruning for Generative Reasoning in Large Language Models
Research demonstrates that layer pruning—a compression technique for large language models—effectively reduces model size while maintaining classification performance, but critically fails to preserve generative reasoning capabilities like arithmetic and code generation. Even with extensive post-training on 400B tokens, models cannot recover lost reasoning abilities, revealing fundamental limitations in current compression approaches.