MR-004 Model & system behaviour System scope

Violent or extremist content generation

The system generates content that incites, glorifies, or facilitates violence, terrorism, or extremism.

Risk family: Model & system behaviour
MIT domain: 1. Discrimination & Toxicity
MIT subdomain: 1.2 > Exposure to toxic content
AI type: GPAI
Scope: System
Source standard: MIT AI Risk Repository v4

Provenance

Source standard

MIT AI Risk Repository v4

MIT source entries

31 entries across 16 papers

Abercrombie2024A Collaborative, Human-Centred Taxonomy of AI, Algorithmic, and Automation Harms58.03.03
Coghlan2023Harm to Nonhuman Animals from AI: a Systematic Account and Framework44.01.00
Critch2023TASRA: a Taxonomy and Analysis of Societal-Scale Risks from AI01.05.00
Deng2023Towards Safer Generative Language Models: A Survey on Safety Risks, Evaluations, and Improvements04.04.00
DSIT2023Capabilities and Risks from Frontier AI67.04.01
Gipiškis2024Risk Sources and Risk Management Measures in Support of Standards for General-Purpose AI Systems62.08.01
IBM2025AI Risk Atlas65.15.05
InfoComm2023Cataloguing LLM Evaluations43.01.01
Li2025A Closer Look at the Existing Risks of Generative AI: Mapping the Who, What, and How of Real-World Incidents66.06.01
Liu2024Trustworthy LLMs: A Survey and Guideline for Evaluating Large Language Models’ Alignment30.02.01
Shelby2023Sociotechnical Harms of Algorithmic Systems: Scoping a Taxonomy for Harm Reduction11.04.02
Stanley2024Emerging Risks and Mitigations for Public Chatbots: LILAC v169.10.00
TC2602024AI Safety Governance Framework45.02.07
Uuk2025A Taxonomy of Systemic Risks from General-Purpose AI61.02.44
Vidgen2024Introducing v0.5 of the AI Safety Benchmark from MLCommons23.01.00 23.01.01 23.01.02 23.01.03 23.01.04 23.01.05 23.02.00 23.02.01 23.02.02 23.02.04 23.07.02
Zeng2024AI Risk Categorization Decoded (AIR 2024): From Government Regulations to Corporate Policies50.02.01 50.02.02 50.02.03 50.02.04 50.02.05 50.02.08

Ev IDs of the entries consolidated into this risk in the MIT AI Risk Repository (V4); the source sheet row appears on hover.

ISO/IEC references

23894 obj A.10; src 7; mech B.5 | 42001 ctrl A.5.4, A.6.2.4

Framework crosswalk

Every framework item mapped to this risk. Items marked partial overlap only in part; definitions appear on hover where the source licence permits.

Sourcesframeworks that contributed to the register

A.10 ISO/IEC 23894 Annex A A.10

A.5.4 ISO/IEC 42001 Annex A A.5.4
A.6.2.4 ISO/IEC 42001 Annex A A.6.2.4

Cross-checksframeworks mapped in to test coverage

AISubtech-15.1.16 Safety Harms and Toxicity: Terrorism / Extremism
AISubtech-15.1.17 Safety Harms and Toxicity: Violence and Public Safety Threat

GENAI.3 Dangerous, Violent, or Hateful Content

More in Model & system behaviour

MR-001 MR-002 MR-003 MR-005 MR-006 MR-007 MR-008 MR-019 MR-021 MR-022 MR-023 MR-025 MR-026 MR-027 MR-028 MR-029 MR-031 MR-049 MR-050 MR-051 MR-052 MR-053 MR-055 MR-056 MR-057 MR-058 MR-059 MR-061

See all Model & system behaviour risks →

Part of the Deployer AI Risk Register, an open-source resource powered by MindXO. Version 1.0, 3 July 2026. Derived from the MIT AI Risk Repository (V4, December 2025) under CC BY 4.0; an independent derivative work, not endorsed by or affiliated with MIT. Sub-risk decomposition references MITRE ATLAS™ v5.6.0 (© 2021-2026 The MITRE Corporation, reproduced and distributed with permission). ISO/IEC and EU AI Act references are by number only. License: CC BY 4.0. Full attribution and licensing.