SkillFactory: Self-Distillation For Learning Cognitive Behaviors

SkillFactory is a novel fine-tuning method that enables language models to learn cognitive behaviors like verification and backtracking without requiring distillation from stronger models. The approach uses self-rearranged training samples during supervised fine-tuning to prime models for subsequent reinforcement learning, resulting in better generalization and robustness.

SkillFactory addresses a fundamental challenge in training reasoning-capable language models: how to equip models with sophisticated cognitive skills when the base model doesn't naturally exhibit them. Traditional approaches rely on distillation from larger or stronger models, which creates dependency on superior systems and limits accessibility. This research demonstrates an alternative pathway using self-distillation, where the model's own outputs are strategically rearranged into training formats that represent desired cognitive behaviors.

The method sits within the broader evolution of reasoning models that employ extended chain-of-thought processing. Recent advances in systems like o1 and similar models show that models can develop reasoning capabilities through appropriate training paradigms, yet the mechanisms enabling skill acquisition remain incompletely understood. SkillFactory contributes empirical evidence that inductive biases established during supervised fine-tuning significantly influence how models subsequently learn to deploy cognitive strategies during reinforcement learning phases.

The research has meaningful implications for the AI development pipeline. Organizations can potentially reduce reliance on larger proprietary models for training smaller, specialized systems. The demonstrated robustness improvements on out-of-domain tasks suggest that silver-quality training traces, while imperfect, provide valuable structural guidance that enhances generalization. This has cost implications for training infrastructure and opens possibilities for more distributed, accessible training methodologies.

Future research should explore scaling this approach to larger model sizes and increasingly complex reasoning domains. The degree to which self-distillation quality degrades with task complexity, and whether the method applies to multimodal or domain-specific models, remain open questions worth investigating.

• →SkillFactory enables self-distillation for cognitive skill learning without requiring stronger teacher models
• →Models trained with SkillFactory SFT show better generalization to harder task variants despite lower pre-RL performance
• →The method demonstrates that imperfect silver training traces can effectively establish useful inductive biases
• →SkillFactory-initialized models exhibit superior robustness on out-of-domain tasks compared to base model baselines
• →Self-rearranged training data from a model's own outputs provides an accessible alternative to external model distillation