Learning CLI Agents with Structured Action Credit under Selective Observation

Researchers present a new approach to training CLI agents through reinforcement learning, introducing σ-Reveal for selective observation and A³ for credit assignment. The work addresses fundamental challenges in teaching AI systems to interact with command-line interfaces by leveraging structured action properties and proposing the ShellOps dataset for evaluation.

This research tackles a critical problem in agent development: teaching AI systems to autonomously interact with command-line interfaces in realistic environments. CLI agents represent a practical frontier for AI-computer interaction, enabling systems to manage filesystems, execute programs, and interpret feedback in real-time. The study identifies two interconnected bottlenecks that have limited progress in this domain.

The selective observation problem addresses a practical reality: codebases and file systems contain vast amounts of information, yet agents receive only partial visibility. σ-Reveal proposes an inference-time mechanism that intelligently filters context within token budgets, enabling agents to focus computational resources on task-relevant evidence. This parallels attention mechanisms in language models but optimized for interactive CLI environments.

The credit assignment challenge relates to how reward signals propagate through long, multi-step trajectories. Traditional reinforcement learning struggles when terminal rewards arrive only after many actions. The proposed A³ method constructs intermediate advantage signals by decomposing episodes into turn-level advantages, using abstract syntax tree analysis to understand action dependencies and trajectory margins. This structured approach preserves computational efficiency while improving learning signal quality.

The introduction of ShellOps as a benchmark dataset addresses evaluation gaps. Having verifiable, standardized tasks enables rigorous comparison of CLI agent approaches and accelerates progress in the field. This work demonstrates how domain-specific structure can be systematically exploited to improve agent learning, with implications extending beyond CLI interactions to any constrained interactive environment where action structure matters.

• →σ-Reveal enables selective observation by intelligently filtering CLI context within token budgets to identify task-relevant information.
• →A³ uses AST-based action decomposition and trajectory margins to construct intermediate advantage signals for long-horizon CLI tasks.
• →ShellOps provides a standardized verifiable dataset suite for evaluating CLI agents on repository-based tasks.
• →The approach preserves algorithmic complexity of standard RL while exploiting native structure of CLI actions.
• →Method addresses practical bottlenecks in training agents for real-world command-line interaction with filesystems and executables.