Memorize Theorems, Not Instances: Probing SFT Generalization through Mathematical Reasoning

Researchers propose Theorem-SFT, a novel supervised fine-tuning approach that teaches language models to apply mathematical rules explicitly rather than memorize surface-level correlations between problems and solutions. The method demonstrates significant performance improvements across benchmarks while revealing that feed-forward layers, not memorization itself, are the primary locus of reasoning capability.

This research addresses a fundamental challenge in large language model training: the brittleness of models fine-tuned on task-specific data. While conventional wisdom attributes generalization failures to memorization, this work reframes the problem as memorizing the wrong targets. Vanilla SFT encourages models to exploit spurious correlations in problem-solution pairs, making them vulnerable to minor input variations—a critical limitation for mathematical and scientific reasoning tasks.

Theorem-SFT redirects the learning objective toward explicit rule application, fundamentally altering what models encode during training. By teaching theorem invocation rather than answer patterns, the approach achieves substantial gains: 8.8% improvement on MATH benchmarks and 20.27% on GeoQA without domain-specific retraining. This cross-domain effectiveness suggests the method captures something more generalizable than surface-level pattern matching.

The architectural findings are particularly significant for the AI community. Fine-tuning only MLP (feed-forward) layers achieves comparable performance to full-network training, suggesting that reasoning rules concentrate in these components while attention mechanisms handle other aspects. This mechanistic insight could streamline future training pipelines and reduce computational costs.

For developers building reasoning-dependent systems, this research implies that training methodology matters as much as data quality. The approach could enhance reliability of AI systems in mathematics, geometry, and logical reasoning—domains where generalization across problem variants is essential. The findings also inform broader discussions about scaling laws and model interpretability, suggesting that how we supervise learning determines not just performance but the robustness of learned capabilities.

• →Theorem-SFT teaches models explicit rule application rather than surface patterns, yielding 8.8-20.27% performance gains across benchmarks.
• →Generalization failures stem from memorizing spurious correlations, not from memorization as a mechanism.
• →Feed-forward MLP layers are the primary locus of reasoning rules, with fine-tuning MLPs alone matching full-network performance.
• →The method generalizes across model families and modalities without domain-specific retraining.
• →Training methodology directly impacts reasoning robustness and generalization to input variations.