Post-Training LLMs as Better Decision-Making Agents: A Regret-Minimization Approach

Researchers introduce Iterative Regret-Minimization Fine-Tuning (Iterative RMFT), a post-training method that improves LLMs' decision-making capabilities by iteratively distilling low-regret trajectories back into models. The approach addresses fundamental limitations in how LLMs handle online decision problems without relying on rigid algorithmic templates, demonstrating improvements across multiple model architectures.

Large language models were engineered primarily for language generation, not sequential decision-making under uncertainty. This fundamental mismatch has created a gap between deployment aspirations and actual performance—LLMs frequently fail at exploration-exploitation tradeoffs and struggle to minimize regret in interactive environments. The Iterative RMFT framework represents a meaningful shift in how researchers approach this problem by leveraging the regret metric as a training signal rather than forcing models into predetermined algorithmic structures.

The technical contribution centers on a feedback loop: models generate multiple decision trajectories, the system ranks them by regret performance, and the model fine-tunes on the best performers. This approach avoids the brittleness of manually crafted chain-of-thought prompts while eliminating dependency on external decision-making algorithms. By allowing models to learn their own reasoning patterns within a principled optimization framework, the method achieves generalization across diverse problem settings—varying time horizons, action spaces, and reward structures.

For the AI systems industry, this work signals movement toward more robust agentic systems. Organizations building AI agents for trading, resource allocation, or operational planning benefit directly from improved decision-making reliability. The empirical validation across model scales (from open-weight to GPT-4o mini) suggests practical accessibility. The theoretical contribution—proving single-layer Transformers can become no-regret learners under this paradigm—provides confidence that the approach rests on solid mathematical foundations rather than empirical hack.

The framework's flexibility makes it particularly valuable for practitioners deploying LLMs in dynamic environments where stakes are high. As LLM-based agents proliferate in finance and operations, methods that systematically improve decision quality become infrastructure-level concerns.

• →Iterative RMFT uses regret minimization as a training signal to improve LLM decision-making without rigid algorithmic templates
• →The method generalizes across diverse model architectures and problem settings with varying horizons and action spaces
• →Model-generated reasoning patterns replace manually crafted prompts, improving flexibility and adaptability
• →Theoretical analysis confirms single-layer Transformers can achieve no-regret learning under this post-training paradigm
• →Framework addresses a critical gap between LLM deployment as agents and their actual performance in interactive environments