From Pixels to Digital Agents: An Empirical Study on the Taxonomy and Technological Trends of Reinforcement Learning Environments

Researchers conducted a large-scale empirical study analyzing over 2,000 publications to map the evolution of reinforcement learning environments. The study reveals a paradigm shift toward two distinct ecosystems: LLM-driven 'Semantic Prior' agents and 'Domain-Specific Generalization' systems, providing a roadmap for next-generation AI simulators.

• →Analysis of 2,000+ publications reveals a fundamental shift in reinforcement learning environment design from isolated simulations to generalist agents.
• →The field has bifurcated into two ecosystems: LLM-dominated 'Semantic Prior' and 'Domain-Specific Generalization' approaches.
• →Researchers developed a novel multi-dimensional taxonomy to systematically analyze RL benchmarks across application domains.
• →The study characterizes 'cognitive fingerprints' to understand cross-task synergy and zero-shot generalization mechanisms.
• →Findings provide a quantitative roadmap for designing Embodied Semantic Simulators that bridge physical control and logical reasoning.