Frontier AI Safety Policies

At the AI Seoul Summit in May 2024, various AI organizations from around the world agreed to the Frontier AI Safety Commitments and to publish a safety framework intended to evaluate and manage severe AI risks.

Published company policies

Anthropic: Responsible Scaling Policy
Published: , Last updated:
OpenAI: Preparedness Framework
Published:
Google DeepMind: Frontier Safety Framework
Published: , Last updated:
Magic: AGI Readiness Policy
Published:
NAVER: AI Safety Framework
Published:
Meta: Frontier AI Framework
Published:
G42: Frontier AI Safety Framework
Published:
Cohere: Secure AI Frontier Model Framework
Published:
Microsoft: Frontier Governance Framework
Published:
Amazon: Frontier Model Safety Framework
Published:
xAI: Risk Management Framework (Draft)
Published:
Nvidia: Frontier AI Risk Assessment
Published:

Other companies

The following organizations have agreed to the Frontier AI Safety Commitments and have not yet published a safety framework, as of writing:

  • 01.AI

  • Inflection AI

  • Minimax

  • Mistral AI

  • Technology Innovation Institute

  • Zhipu AI

IBM has summarized its existing AI governance practices without publishing a safety framework for managing severe AI risks. Samsung has published an on-device AI Safety Framework.

Our indexing these documents should not be considered an endorsement of their substance. Our hope is that making what has been published accessible enables more dialogue, comparison, and critique.

Resources

METR – Common Elements of Frontier AI Safety Policies

How frontier AI companies are using evaluations to understand and manage emerging risks from their systems.
Frontier Model Forum – Components of Frontier AI Safety Frameworks

Issue brief listing components of frontier AI safety frameworks.
Carnegie – A Sketch of Potential Tripwire Capabilities for AI

Paper summarizing AI capability thresholds, safety and security mitigations, and evaluations.
Carnegie – DeepSeek and Other Chinese Firms Converge with Western Companies on AI Promises

Article on how China's AI Safety Commitments mirror those of the Frontier AI Safety Commitments.
Federation of American Scientists – Can Preparedness Frameworks Pull Their Weight?

Article on shortcomings of current frameworks, including underspecified risk thresholds and insufficiently conservative risk mitigations.

METR has written example framework drafts, such as AI Safety Framework for Pre-Frontier AI and Example Safety and Security Framework. These are illustrative only and do not represent recommendations.

We are available to support frontier AI labs exploring frontier AI safety policies. If interested in a conversation, reach out at info@metr.org.