AIBullisharXiv – CS AI · Jun 237/10
🧠Researchers introduce DEAR, a novel on-policy distillation method that improves AI model training by distinguishing between decision tokens (where models branch) and evidence tokens (supporting intermediate steps). The technique achieves significant performance gains of up to 5.7% on code generation and 2.5% on math benchmarks compared to standard distillation approaches.
AIBullisharXiv – CS AI · May 277/10
🧠Researchers propose MUSE-Autoskill, a framework enabling LLM agents to autonomously create, store, and refine reusable skills throughout their operational lifecycle. The system treats skills as long-lived, testable assets with integrated memory and evaluation mechanisms, demonstrating improved task success rates and cross-agent knowledge transfer on benchmark tests.
AIBullisharXiv – CS AI · May 127/10
🧠Researchers introduce SPARK, a framework that verifies AI agent skills through direct environment interaction rather than relying on pre-written plans. The Posterior Distillation Index (PDI) metric ensures skills are grounded in actual task evidence, producing student models that match or exceed human-written skills while reducing inference costs by up to 1,000x.
AIBullisharXiv – CS AI · May 127/10
🧠Researchers introduce Priming, a method that converts pre-trained Transformers into efficient Hybrid State-Space models through knowledge transfer rather than training from scratch. The technique recovers downstream performance using less than 0.5% of original pre-training tokens and enables the first large-scale comparison of SSM architectures, with Hybrid GKA 32B achieving 3.8-point reasoning improvements while delivering 2.3x faster decoding.
🧠 Llama
AIBullisharXiv – CS AI · May 97/10
🧠Researchers introduce LANTERN, a framework that uses large language models to automatically generate task descriptions and intelligently aggregate knowledge from multiple source tasks for reinforcement learning. The system achieves 40-60% improvements in sample efficiency by adaptively weighting source policies based on task similarity and managing teacher-student knowledge transfer through uncertainty-aware gating.
AIBullisharXiv – CS AI · May 97/10
🧠Researchers introduce Stellar VLA, a continual learning framework for vision-language-action models that improves knowledge accumulation without adding network parameters. The approach uses knowledge-guided expert routing and hierarchical task structures, achieving strong performance on robotics benchmarks with minimal data replay and validated real-world transfer capabilities.
AIBullisharXiv – CS AI · Apr 157/10
🧠Researchers propose a case-based learning framework enabling LLM-based autonomous agents to extract and reuse knowledge from past tasks, improving performance on complex real-world problems. The method outperforms traditional zero-shot, few-shot, and prompt-based baselines across six task categories, with gains increasing as task complexity rises.
AIBullisharXiv – CS AI · Apr 157/10
🧠Researchers present Chain-of-Models Pre-Training (CoM-PT), a novel method that accelerates vision foundation model training by up to 7.09X through sequential knowledge transfer from smaller to larger models in a unified pipeline, rather than training each model independently. The approach maintains or improves performance while significantly reducing computational costs, with efficiency gains increasing as more models are added to the training sequence.
AIBullisharXiv – CS AI · Feb 277/106
🧠Researchers introduce GraftLLM, a new method for transferring knowledge between large language models using 'SkillPack' format that preserves capabilities while avoiding catastrophic forgetting. The approach enables efficient model fusion and continual learning for heterogeneous models through modular knowledge storage.
AINeutralarXiv – CS AI · Jun 196/10
🧠Researchers propose MENTOR, a reinforcement learning framework that improves how small language models learn tool-use capabilities from larger models by using flexible, process-aware rewards instead of rigid trajectory replication. The approach demonstrates better out-of-domain generalization than supervised fine-tuning and strict RL baselines in executable-tool environments.
AINeutralarXiv – CS AI · Jun 116/10
🧠Researchers identify a critical failure mode in on-policy distillation where reintroducing privileged context (like system prompts) to a distilled student model degrades performance, even on previously solved tasks. They propose a lightweight consistency regularizer using stop-gradient anchoring and forward KL divergence to achieve 'context removability,' enabling models to internalize context while remaining stable when it reappears.
AINeutralarXiv – CS AI · Jun 116/10
🧠Researchers propose interleaved stacking, a novel training method for distilling large speech foundation models into efficient student models while accelerating training speed. The technique maintains consistent layer positions during progressive depth expansion, addressing performance degradation issues in existing stacking approaches and demonstrating effectiveness on the SUPERB benchmark.
AINeutralarXiv – CS AI · Jun 106/10
🧠Researchers propose MoE-FedTP, a federated learning framework using Mixture-of-Experts networks to improve traffic prediction across cities while preserving privacy. The system enables data-rich cities to share knowledge with data-scarce regions by dynamically fusing expert networks tailored to different urban environments, achieving superior accuracy without centralized data collection.
AIBullisharXiv – CS AI · Jun 86/10
🧠Researchers propose a novel framework using Large Language Models and Retrieval-Augmented Generation to address the cold-start problem in multi-vertical e-commerce platforms by transferring behavioral knowledge from data-rich verticals like restaurants to emerging categories like grocery and retail. The approach synthesizes hierarchical taxonomic features from user order histories and integrates them into a Multi-Task Learning ranking model, demonstrating improved personalization in production environments.
AINeutralarXiv – CS AI · Jun 26/10
🧠Researchers introduce AgentCL, an evaluation framework for assessing continual learning in language agents, along with MemProbe, a memory design method that helps agents accumulate and reuse knowledge across tasks while avoiding interference. The framework uses controlled task streams to rigorously measure how well agents learn and transfer knowledge over time, revealing that current memory designs struggle to balance learning plasticity with stable knowledge reuse.
AINeutralarXiv – CS AI · Jun 26/10
🧠Researchers demonstrate that temperature scaling fundamentally alters the performance comparison between forward KL and reverse KL divergence in LLM distillation, revealing that forward KL substantially outperforms reverse KL at higher temperatures by better leveraging non-dominant token signals. This finding challenges the prevailing preference for reverse KL and suggests that temperature optimization enables simple KL-based methods to match state-of-the-art distillation approaches.
AINeutralarXiv – CS AI · Jun 26/10
🧠Researchers present a unified theoretical framework analyzing knowledge transfer (KT) in machine learning through spectral analysis of SGD dynamics. The study reveals two distinct mechanisms—Spectral Horizon Expansion in knowledge distillation and Spectral Denoising in weak-to-strong generalization—explaining how knowledge transfer efficiency is governed by implicit regularization and heterogeneous spectral learning speeds.
AINeutralarXiv – CS AI · May 126/10
🧠Researchers evaluate LLM-guided semi-supervised learning methods for classifying crisis-related social media data, finding that LG-CoTrain significantly outperforms traditional approaches in low-resource settings while compact models can rival large zero-shot LLMs. This demonstrates practical pathways for deploying AI in disaster response applications with minimal labeled training data.
AIBullisharXiv – CS AI · May 126/10
🧠Researchers introduced DiffKT3D, a 3D diffusion model framework that applies knowledge transfer from video diffusion models to radiotherapy dose prediction. The approach achieves state-of-the-art results by reducing prediction error by 7% compared to previous benchmarks while maintaining clinical alignment through reinforcement learning post-training.
AIBullisharXiv – CS AI · May 116/10
🧠Researchers demonstrate that different 3D medical imaging domains (CT, MRI, PET) transfer knowledge asymmetrically during pretraining, following predictable power-law patterns. By optimizing data allocation based on these transfer dynamics, they achieve up to 58% performance gains over proportional sampling, revealing a hub-and-island structure where certain domains act as foundational knowledge sources for others.
AIBullisharXiv – CS AI · Apr 66/10
🧠Researchers developed new compression techniques for LLM-generated text, achieving massive compression ratios through domain-adapted LoRA adapters and an interactive 'Question-Asking' protocol. The QA method uses binary questions to transfer knowledge between small and large models, achieving compression ratios of 0.0006-0.004 while recovering 23-72% of capability gaps.
AIBullisharXiv – CS AI · Mar 266/10
🧠Researchers developed Med-Shicheng, a framework that enables lightweight LLMs to learn and transfer medical expertise from distinguished physicians. Built on a 1.5B parameter model, it achieves performance comparable to much larger models like GPT-5 while running on resource-constrained hardware.
🧠 GPT-5
AIBullisharXiv – CS AI · Mar 176/10
🧠Researchers propose DeLL, a new framework for autonomous driving systems that addresses lifelong learning challenges through dynamic knowledge spaces and causal inference mechanisms. The system uses Dirichlet process mixture models to prevent catastrophic forgetting and improve adaptability to new driving scenarios while maintaining previously learned knowledge.
AINeutralarXiv – CS AI · Mar 166/10
🧠This comprehensive survey examines continual learning methodologies for large language models, focusing on three core training stages and methods to mitigate catastrophic forgetting. The research reveals that while current approaches show promise in specific domains, fundamental challenges remain in achieving seamless knowledge integration across diverse tasks and temporal scales.
AIBullisharXiv – CS AI · Mar 126/10
🧠Researchers introduce HEAL (Hindsight Entropy-Assisted Learning), a new framework for distilling reasoning capabilities from large AI models into smaller ones. The method overcomes traditional limitations by using three core modules to bridge reasoning gaps and significantly outperforms standard distillation techniques.
🏢 Perplexity
