MR-053 Model & system behaviour System scope

Goal misspecification, specification gaming and reward hacking

The system optimizes a misspecified proxy objective, gaming the specification or misgeneralizing its goal in deployment.

Risk family: Model & system behaviour
MIT domain: 7. AI System Safety, Failures, & Limitations
MIT subdomain: 7.1 > AI pursuing its own goals in conflict with human goals or values
AI type: GPAI, Agentic, Classical_ML
Scope: System
Source standard: MIT AI Risk Repository v4

Provenance

Source standard

MIT AI Risk Repository v4

MIT source entries

41 entries across 19 papers

Anwar2024Foundational Challenges in Assuring Alignment and Safety of Large Language Models73.01.02
Clarke2023A Survey of the Potential Long-term Impacts of AI: How AI Could Lead to Long-term Changes in Science, Cooperation, Power, Epistemics and Values55.05.00 55.05.01 55.05.02
G'sell2024Regulating under Uncertainty: Governance Options for Generative AI47.01.03
Gabriel2024The Ethics of Advanced AI Assistants24.02.00 24.02.02 24.02.03
Giarmoleo2024What Ethics Can Say on Artificial Intelligence: Insights from a Systematic Literature Review37.02.01
Gipiškis2024Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems62.22.01 62.22.02 62.22.03 62.22.04
Hammond2025Multi-Agent Risks from Advanced AI63.09.02
Hendrycks2022X-Risk Analysis for AI Research35.04.00
Hendrycks2023An Overview of Catastrophic AI Risks22.04.01 22.04.03
Hogenhout2021A framework for ethical Ai at the United Nations06.08.00
Ji2023AI Alignment: A Comprehensive Survey34.01.01 34.01.02 34.01.03 34.02.00 34.03.00 34.03.05
Kilian2023Examining the differential risk from high-level artificial intelligence and the question of control07.03.00
Leech2024Ten Hard Problems in Artificial Intelligence We Must Get Right54.03.01
Maas2023Advancing AI Governance: A Literature Review of Problems, Options, and Proposals53.01.00 53.01.01 53.01.02 53.01.03 53.01.04 53.01.05 53.01.06 53.01.07 53.01.08
Saghiri2022A Survey of Artificial Intelligence Challenges: Analyzing the Definitions, Relationships, and Evolutions39.26.00
Steimers2022Sources of Risk of AI Systems14.03.00
Uuk2025A Taxonomy of Systemic Risks from General-Purpose AI61.02.30
Yampolskiy2016Taxonomy of Pathways to Dangerous Artificial Intelligence40.03.00 40.04.00
Zhang2022Towards risk-aware artificial intelligence and machine learning systems: An overview21.02.01.a

Ev IDs of the entries consolidated into this risk in the MIT AI Risk Repository (V4); the source sheet row appears on hover.

ISO/IEC references

23894 obj A.9; src 7; mech B.5 | 42001 ctrl A.6.2.4

Framework crosswalk

Every framework item mapped to this risk. Items marked partial overlap only in part; definitions appear on hover where the source licence permits.

Sourcesframeworks that contributed to the register

A.9 ISO/IEC 23894 Annex A A.9

A.6.2.4 ISO/IEC 42001 Annex A A.6.2.4

More in Model & system behaviour

MR-001 MR-002 MR-003 MR-004 MR-005 MR-006 MR-007 MR-008 MR-019 MR-021 MR-022 MR-023 MR-025 MR-026 MR-027 MR-028 MR-029 MR-031 MR-049 MR-050 MR-051 MR-052 MR-055 MR-056 MR-057 MR-058 MR-059 MR-061

See all Model & system behaviour risks →

Part of the Deployer AI Risk Register, an open-source resource powered by MindXO. Version 1.0, 3 July 2026. Derived from the MIT AI Risk Repository (V4, December 2025) under CC BY 4.0; an independent derivative work, not endorsed by or affiliated with MIT. Sub-risk decomposition references MITRE ATLAS™ v5.6.0 (© 2021-2026 The MITRE Corporation, reproduced and distributed with permission). ISO/IEC and EU AI Act references are by number only. License: CC BY 4.0. Full attribution and licensing.