14 articles tagged with #supervised-fine-tuning. AI-curated summaries with sentiment analysis and key takeaways from 50+ sources.
Researchers introduce LLUMI, an open-source LLM system for mental health support that uses community feedback from Reddit to improve response quality without relying on proprietary cloud models. The approach achieves comparable performance to GPT models while offering better privacy protection for sensitive health contexts.
Researchers propose Theorem-SFT, a novel supervised fine-tuning approach that teaches language models to apply mathematical rules explicitly rather than memorize surface-level correlations between problems and solutions. The method demonstrates significant performance improvements across benchmarks while revealing that feed-forward layers, not memorization itself, are the primary locus of reasoning capability.
Researchers propose a new framework called On-Policy SFT that bridges the performance gap between supervised fine-tuning and reinforcement learning in AI model training. The framework introduces Distribution Discriminant Theory (DDT) and two techniques - In-Distribution Finetuning and Hinted Decoding - that achieve better generalization while maintaining computational efficiency.
Researchers developed a new AI training method using knowledge graphs as reward models to improve compositional reasoning in specialized domains. The approach enables smaller 14B parameter models to outperform much larger frontier systems like GPT-5.2 and Gemini 3 Pro on complex multi-hop reasoning tasks in medicine.
Researchers introduce GraphSSR, a new framework that improves zero-shot graph learning by combining Large Language Models with adaptive subgraph denoising. The system addresses structural noise issues in existing methods through a dynamic 'Sample-Select-Reason' pipeline and reinforcement learning training.
Researchers propose a new approach for training AI models to generate correct answers from demonstrations, using imitation learning in contextual bandits rather than traditional supervised fine-tuning. The method achieves better sample complexity and works with weaker assumptions about the underlying reward model compared to existing likelihood-maximization approaches.
Researchers identify harmful continuation in long chain-of-thought training data where LLMs continue reasoning after the answer is sufficiently supported, degrading fine-tuning performance. Using a delete-only editor, they remove post-conclusion continuations and demonstrate improved SFT outcomes, introducing Harmful Continuation Cut (HCC) as a lightweight solution to detect and eliminate this problematic pattern.
Researchers present a novel framework analyzing how reinforcement learning (RL) and supervised fine-tuning (SFT) differently shape reasoning in large language models. The study reveals that RL compresses incorrect reasoning paths while SFT expands correct ones, explaining why the two-stage training approach produces superior reasoning capabilities across models of 1.5B to 14B parameters.
Researchers demonstrate that reinforcement learning can synthesize novel compositional reasoning skills, but only when models first master independent atomic skills through supervised fine-tuning. Using a controlled synthetic dataset, they show SFT alone produces memorization without generalization, while RL bridges the gap to genuine skill integration when prerequisites are met.
Researchers prove that supervised fine-tuning (SFT) and reinforcement learning (RL) cannot be decoupled during large language model post-training, as each method degrades the performance gains of the other. The theoretical findings, verified experimentally, challenge the widespread industry practice of alternating these two training approaches and suggest optimal RL duration exists to balance competing objectives.
Researchers introduce SAI-DPO, a dynamic data sampling framework that adapts training data selection based on a model's evolving capabilities during training, rather than using static metrics. Tested on mathematical reasoning benchmarks including AIME24 and AMC23, SAI-DPO achieves state-of-the-art performance with significantly less training data, outperforming baselines by nearly 6 points.
Researchers present a layer-wise analysis of Supervised Fine-Tuning (SFT) in large language models, revealing that middle layers remain stable during training while final layers exhibit high sensitivity. They introduce Mid-Block Efficient Tuning, a targeted approach that selectively updates intermediate layers and achieves up to 10.2% performance gains over standard LoRA on benchmarks with significantly reduced parameter overhead.
A comprehensive research study examines the relationship between Supervised Fine-Tuning (SFT) and Reinforcement Learning (RL) methods for improving Large Language Models after pre-training. The research identifies emerging trends toward hybrid post-training approaches that combine both methods, analyzing applications from 2023-2025 to establish when each method is most effective.
New theoretical research analyzes how Large Language Models learn during pretraining versus post-training phases, revealing that balanced pretraining data creates latent capabilities activated later, while supervised fine-tuning works best on small, challenging datasets and reinforcement learning requires large-scale data that isn't overly difficult.
