20 articles tagged with #offline-learning. AI-curated summaries with sentiment analysis and key takeaways from 50+ sources.
Researchers achieved breakthrough sample complexity improvements for offline reinforcement learning algorithms using f-divergence regularization, particularly for contextual bandits. The study demonstrates optimal O(ε⁻¹) sample complexity under single-policy concentrability conditions, significantly improving upon existing bounds.
Researchers introduce Chain-of-Goals Hierarchical Policy (CoGHP), a novel framework that applies chain-of-thought reasoning to offline reinforcement learning by autoregressively generating sequences of intermediate subgoals to solve long-horizon tasks. The unified architecture demonstrates consistent performance improvements over existing hierarchical baselines on navigation and manipulation benchmarks.
Researchers propose SOCD, an offline reinforcement learning algorithm that learns multi-user scheduling policies from pre-collected data without requiring real-time system interactions. The method combines diffusion models with critic guidance and Lagrangian optimization to handle delay-constrained resource allocation across applications like data centers and live streaming.
Researchers introduce Model Predictive Diffuser (MPDiffuser), a diffusion-based framework for offline decision-making that combines trajectory planning with dynamics modeling to generate more reliable and feasible control sequences. The approach shows consistent improvements over existing diffusion methods across benchmark tasks and demonstrates real-world viability through robot deployment.
Researchers propose Safe-RULE, a new reinforcement unlearning framework designed to defend offline safe reinforcement learning systems against data poisoning attacks. The approach removes malicious data influence without requiring model retraining or access to original training environments, addressing a critical vulnerability in safety-critical applications like robotics.
Researchers evaluated the AIcon2abs method, an educational framework using the WiSARD weightless neural network algorithm to teach machine learning concepts to diverse audiences from K-12 students to adults. A six-hour remote course with 34 Brazilian participants demonstrated high satisfaction rates, with the approach enabling intuitive understanding of ML training and classification through hands-on activities without requiring internet connectivity.
Researchers propose Dual Advantage Fields (DAF), a reinforcement learning method that extracts local policy signals from dual value representations to improve offline goal-conditioned learning. The approach combines global reachability estimates with local action preferences, showing strong performance on locomotion, manipulation, and puzzle tasks where direct movement toward goals isn't optimal.
Researchers introduce DEFLECT, an offline post-training framework that improves Vision-Language-Action (VLA) robot policies by addressing latency-induced misalignment in asynchronous inference. The method uses counterfactual preference learning to teach policies to favor execution-time-aligned actions over stale prediction-time actions, achieving up to 6.4 percentage-point improvements in high-latency success rates without requiring human labels, reward models, or architectural changes.
Researchers introduce EmoDistill, an offline framework that teaches language model agents to strategically use emotion in adversarial negotiations. The system decomposes emotional strategy into emotion selection and expression, with experiments showing that emotionally-framed language significantly shifts negotiation outcomes, suggesting emotion functions as a tactical tool rather than stylistic decoration.
Researchers demonstrate that large language models can be effectively fine-tuned to perform sequential decision-making tasks across MDPs, POMDPs, and ambiguous environments by learning from offline trajectory data. The approach achieves stronger performance than baseline methods, particularly in complex, partially-observed scenarios, with theoretical analysis showing the fine-tuned attention mechanisms implicitly estimate optimal Q-functions.
Researchers achieve the first fast statistical rates (Õ(ε⁻¹)) for offline contextual bandits using forward-KL regularization under single-policy concentrability, matching the performance previously only shown for reverse-KL approaches and establishing rate-optimal lower bounds.
Researchers introduce WOMBET, a framework that improves reinforcement learning efficiency in robotics by generating synthetic training data from a world model in source tasks and selectively transferring it to target tasks. The approach combines offline-to-online learning with uncertainty-aware planning to reduce data collection costs while maintaining robustness.
Researchers have developed OPRIDE, a new algorithm for offline preference-based reinforcement learning that significantly improves query efficiency. The algorithm addresses key challenges of inefficient exploration and overoptimization through principled exploration strategies and discount scheduling mechanisms.
Researchers propose OM2P, a new offline multi-agent reinforcement learning algorithm that achieves efficient one-step action sampling using mean-flow models. The approach delivers up to 3.8x reduction in GPU memory usage and 10.8x speed-up in training time compared to existing diffusion and flow-based models.
Researchers developed an offline-to-online reinforcement learning framework that improves robot control robustness through adversarial fine-tuning. The method trains policies on clean datasets then applies action perturbations during fine-tuning to build resilience against actuator faults and environmental uncertainties.
Researchers have developed LLM4Cov, an offline learning framework that enables AI agents to generate high-coverage hardware verification testbenches without expensive online reinforcement learning. A compact 4B-parameter model achieved 69.2% coverage pass rate, outperforming larger models by demonstrating efficient learning from execution feedback in hardware verification tasks.
Researchers introduce Safe Flow Q-Learning (SafeFQL), a new offline safe reinforcement learning method that combines Hamilton-Jacobi reachability with flow policies for safety-critical real-time control. The method achieves better safety performance with lower inference latency compared to existing diffusion-based approaches, making it more suitable for real-time deployment.
The article discusses a model-based control approach for efficient learning and exploration that combines online planning with offline learning. This methodology aims to optimize the balance between computational efficiency and learning effectiveness in AI systems.
Researchers developed COffeE-PSRO, a new algorithm that applies offline reinforcement learning to game-theoretic multiagent systems. The approach extends Policy Space Response Oracles by incorporating uncertainty quantification and conservative exploration to find equilibrium strategies from fixed datasets without online interaction.
Researchers propose OVMSE, a new framework for Offline-to-Online Multi-Agent Reinforcement Learning that addresses key challenges in transitioning from offline training to online fine-tuning. The framework introduces Offline Value Function Memory and Sequential Exploration strategies to improve sample efficiency and performance in multi-agent environments.
