What Capable Agents Must Know: Selection Theorems for Robust Decision-Making under Uncertainty

Researchers prove 'selection theorems' showing that AI agents achieving low regret on prediction tasks must develop internal predictive models and belief states. The work demonstrates that structured internal representations are mathematically necessary, not just helpful, for competent decision-making under uncertainty.

• →Mathematically proves that capable AI agents must develop predictive internal models to achieve low regret on structured prediction tasks.
• →Shows that belief-like memory and predictive state representations are necessary requirements, not optional features, for agents under partial observability.
• →Covers stochastic policies and partial observability without requiring optimality assumptions or explicit model access.
• →Addresses fundamental questions about what internal structure AI agents need to act competently under uncertainty.
• →Uses novel technical approach reducing predictive modeling to binary betting decisions with regret bounds.