Consequentialist Objectives and Catastrophe

A research paper argues that advanced AI systems with fixed consequentialist objectives will inevitably produce catastrophic outcomes due to their competence, not incompetence. The study establishes formal conditions under which such catastrophes occur and suggests that constraining AI capabilities is necessary to prevent disaster.