GIFT: LLM-Guided State-Reward Interface for Financial Reinforcement Learning

Researchers introduce GIFT, an LLM-guided framework that enhances reinforcement learning for portfolio trading by using language models to design better state features and reward signals rather than making trading decisions directly. The approach combines factor-guided state enhancement, risk-rule-guided reward shaping, and diagnostic refinement to improve out-of-sample portfolio performance across diverse market conditions.

GIFT represents a sophisticated application of large language models to a fundamental challenge in quantitative finance: designing effective learning interfaces for reinforcement learning agents in non-stationary markets. The framework addresses a critical gap where traditional raw market data (OHLCV) and short-term return signals prove insufficient for training robust trading agents, particularly during regime shifts and market volatility.

The approach departs from the increasingly common tendency to use LLMs as direct decision-makers in financial applications. Instead, GIFT leverages LLM capabilities where they excel—incorporating domain expertise into feature engineering and reward design—while maintaining the computational efficiency and interpretability of deterministic policies after the initialization phase. This hybrid architecture reduces inference costs and eliminates the risk of LLM hallucinations during live trading.

The three-stage methodology demonstrates thoughtful system design: Factor-guided State Enhancement generates meaningful financial features from interpretable primitives, Risk-rule-guided Reward Shaping injects portfolio risk constraints that historically matter to practitioners, and Diagnostic-guided Refinement uses PPO rollout diagnostics to validate interface choices empirically. Crucially, the framework fixes selected interfaces before evaluation, preventing information leakage and ensuring fair backtesting.

The significance extends beyond academic interest. Portfolio managers increasingly explore learning-based approaches, and better state-reward interfaces directly translate to more reliable risk-adjusted returns. The rolling-window experiments across multiple market regimes suggest genuine generalization rather than overfitting, though the practical performance advantages over simpler baselines warrant scrutiny from institutional traders implementing algorithmic strategies.

• →GIFT uses LLMs to design state and reward representations rather than make trading decisions, reducing inference costs and hallucination risks
• →The framework combines financial factors, risk rules, and empirical diagnostics to improve learning-signal quality in portfolio optimization
• →Fixed state-reward interfaces after refinement prevent overfitting and enable genuine out-of-sample evaluation
• →Rolling-window experiments demonstrate improved risk-adjusted performance across diverse market regimes
• →The approach addresses a core challenge in applying RL to financial trading: designing effective learning interfaces for non-stationary markets