AutoTool: Automatic Scaling of Tool-Use Capabilities in RL via Decoupled Entropy Constraints

Researchers introduce AutoTool, a new reinforcement learning approach that enables AI agents to automatically scale their reasoning capabilities for tool use. The method uses entropy-based optimization and supervised fine-tuning to help models efficiently determine appropriate thinking lengths for simple versus complex problems, achieving 9.8% accuracy improvements while reducing computational overhead by 81%.

• →AutoTool addresses key challenges in current RL-based scaling approaches for AI tool use by preventing overthinking on simple problems.
• →The method combines warm-up supervised fine-tuning with entropy-based RL to help models distinguish between simple and complex problems.
• →Entropy-based optimization maintains model diversity while successfully unlocking automatic scaling capabilities.
• →Testing on three benchmarks showed 9.8% accuracy improvements with approximately 81% reduction in computational overhead.
• →The approach enables AI agents to automatically determine appropriate reasoning trajectory lengths for efficient tool use.