On the Limits of Layer Pruning for Generative Reasoning in Large Language Models

Research demonstrates that layer pruning—a compression technique for large language models—effectively reduces model size while maintaining classification performance, but critically fails to preserve generative reasoning capabilities like arithmetic and code generation. Even with extensive post-training on 400B tokens, models cannot recover lost reasoning abilities, revealing fundamental limitations in current compression approaches.

Layer pruning has emerged as a promising technique for reducing the computational footprint of LLMs, with prior research showing minimal performance loss on traditional benchmarks. However, this study reveals a stark divergence: while classification tasks recover up to 90% of baseline performance through supervised finetuning, generative reasoning tasks demonstrate persistent degradation that resists recovery even under substantial post-training investment.

The findings expose a critical vulnerability in model compression strategies. When layers are removed, LLMs lose not merely parameters but encoded algorithmic capabilities—the ability to perform arithmetic operations and maintain syntactic constraints like balanced parentheses. These losses appear structural rather than superficial, suggesting that certain reasoning abilities are distributed across depth in ways that compression disrupts irreversibly.

For the AI industry, this research challenges assumptions about model efficiency gains. Organizations pursuing deployment optimization must now account for reasoning task performance separately from classification metrics. The inability to recover capabilities despite 400B tokens of finetuning—orders of magnitude more than typical post-training—indicates fundamental architectural constraints rather than insufficient fine-tuning.

This work directly impacts model deployment decisions for reasoning-heavy applications, particularly in code generation, mathematics, and logical reasoning domains. Teams cannot simply compress models expecting recovery through finetuning; instead, they must evaluate reasoning performance explicitly before deployment. Future research likely focuses on alternative compression methods preserving depth-dependent reasoning abilities, or developing hybrid approaches that selectively compress non-critical layers while preserving reasoning pathways.

• →Layer pruning recovers classification performance but fundamentally fails to restore generative reasoning and arithmetic capabilities
• →Even 400B tokens of post-training finetuning cannot recover lost reasoning performance, indicating structural rather than superficial capability loss
• →Arithmetic computation and syntactic constraint maintenance appear distributed across model depth in ways compression permanently disrupts
• →Organizations must evaluate reasoning task performance explicitly; compression optimization cannot assume across-the-board recovery
• →Current compression techniques prove insufficient for reasoning-heavy applications, driving need for alternative depth-preservation methods