Answer-Set-Programming-based Abstractions for Reinforcement Learning

Researchers have developed an Answer-Set Programming (ASP) based implementation of the CARCASS framework to improve Reinforcement Learning abstractions for complex state spaces. The approach leverages ASP's declarative modeling capabilities as an alternative to Prolog, demonstrating promising results in Blocks World and Minigrid domains when domain knowledge is available.

This research addresses a fundamental challenge in reinforcement learning: scaling algorithms to handle exponentially large state spaces. Traditional RL approaches struggle with realistic problems because they must learn policies across vast solution spaces. The CARCASS framework, originally developed by Martijn van Otterlo, tackles this by using logical representations to construct meaningful abstractions of Markov Decision Processes in first-order domains.

The key innovation here is the shift from Prolog to Answer-Set Programming. While Prolog relies on procedural execution through backtracking, ASP provides a fully declarative framework where programmers specify what should be true rather than how to compute it. This declarative nature aligns better with domain modeling requirements and potentially offers cleaner integration of domain knowledge into the learning process.

For the AI research community, this work bridges symbolic AI reasoning with modern reinforcement learning, two paradigms that have historically developed separately. By using relational representations, the approach can reason about objects and their relationships explicitly, enabling better generalization across similar problem instances. The evaluation on Blocks World—a classical AI planning domain—and Minigrid demonstrates practical viability in structured environments.

The broader implications extend to embodied AI systems and robotics, where domain knowledge is often available and state spaces are naturally relational. As RL agents become more complex and specialized, human-guided abstraction through declarative specifications could become increasingly valuable. Future work should examine scalability to larger domains and comparison against contemporary abstraction learning techniques.

• →Answer-Set Programming provides a more declarative alternative to Prolog for implementing CARCASS abstractions in reinforcement learning.
• →The approach leverages relational reasoning to construct meaningful abstractions that reduce state space complexity.
• →Successful evaluation on Blocks World and Minigrid demonstrates practical viability in structured domains.
• →This work bridges symbolic AI and modern RL by incorporating explicit domain knowledge into policy learning.
• →The method shows particular promise when domain knowledge is available, limiting applicability in fully unstructured environments.