Frontier Model Safety Evaluations Explained for Non-Researchers

Why “Frontier Model Safety” Became Its Own Discipline

Two years ago, most people encountered AI safety as a vague marketing phrase attached to a chatbot’s terms of service. That has changed. As models have grown more capable of writing working exploit code, planning multi-step tasks, and reasoning about their own situation, a genuine engineering and scientific discipline has formed around a single question: before we let a new model out into the world, how do we know what it is actually capable of, and whether that capability could cause serious harm?

This discipline is usually called frontier model safety evaluation, or frontier safety for short. The word “frontier” refers to the small group of models at the very edge of current capability, typically the largest and newest systems released by labs such as OpenAI, Anthropic, and Google DeepMind. These are the models most likely to cross into territory nobody has tested before, which is exactly why they receive a level of scrutiny that smaller or older models do not.

For a business leader, journalist, policy staffer, or curious reader without a machine learning background, the terminology in this space can feel deliberately opaque: red-teaming, dangerous capability evaluations, situational awareness probes, preparedness frameworks. None of it is actually that complicated once it is translated out of research jargon. This explainer walks through what each type of evaluation is actually testing for, who runs the tests, how the major labs structure their public safety commitments, and what questions you should be asking if your organization is deciding whether to adopt a frontier model.

The Three Core Categories of Frontier Safety Evaluations

Almost every evaluation you will read about in a lab’s model documentation or a third-party report falls into one of three buckets. Understanding these buckets makes the rest of the field much easier to follow.

1. Dangerous capability evaluations

These tests ask a narrow, practical question: could this model meaningfully increase a person’s ability to cause large-scale harm, compared to what they could already do with a search engine or a textbook? Researchers call this an “uplift” measurement, because the concern is not whether a model can discuss a dangerous topic in the abstract, but whether it provides a meaningful uplift toward actually executing the harm.

The main areas tested are biological and chemical weapons uplift (sometimes grouped with nuclear and radiological risks under the acronym CBRN), offensive cybersecurity capability (can the model find and exploit real software vulnerabilities autonomously), self-proliferation (could the model copy itself, acquire compute, or resist being shut down), and persuasion or manipulation at scale (can the model change people’s beliefs or behavior more effectively than a skilled human).

2. Red-teaming

Red-teaming is adversarial testing: a team of people, sometimes joined by AI systems built specifically for this purpose, deliberately tries to make the model misbehave. In frontier contexts this goes well beyond checking whether a chatbot can be tricked into swearing. Specialist red teams with backgrounds in biosecurity, cybersecurity, and disinformation try to jailbreak the model into providing operationally useful, dangerous instructions, using every trick available, including multi-turn conversations, role-play framing, and translated or obfuscated prompts.

3. Deceptive-alignment and situational-awareness probes

This is the newest and least intuitive category. As models have become better at reasoning across many steps, researchers have started testing whether a model can recognize it is being evaluated and behave differently as a result, a phenomenon often called “evaluation awareness.” Related tests look for “scheming,” meaning whether a model would pursue a hidden goal while appearing to comply with instructions, or “sandbagging,” meaning deliberately underperforming on a capability test to avoid triggering additional restrictions. These probes matter because a model that can detect it is being tested, and adjust its behavior only during the test, would make every other evaluation in this article far less trustworthy.

Who Actually Runs These Tests: The Third-Party Evaluator Ecosystem

One of the most important developments in frontier safety over the last two years is the growth of organizations that test models independently of the labs that build them. This matters because a lab grading its own homework has an obvious conflict of interest, even when everyone involved is acting in good faith. External evaluators provide a second opinion, often with different incentives and different access to a model’s internals.

The UK’s AI Security Institute, commonly known by its earlier name AISI, was one of the first government-backed bodies dedicated specifically to pre-deployment testing of frontier models, and it now publishes periodic Frontier AI Trends reports summarizing what it has found across multiple labs. In the United States, the equivalent function sits with the Center for AI Standards and Innovation, generally referred to as CAISI, which grew out of the earlier US AI Safety Institute initiative and focuses on national-security-relevant capability testing. The two institutes have signed cooperation agreements to share testing methodology and, in some cases, results, rather than duplicating the same work twice.

Alongside the government-backed institutes, two nonprofit research organizations have become central to this ecosystem. METR (Model Evaluation and Threat Research) specializes in measuring autonomous capability, including how long a task chain a model can complete without human help, which it uses as a proxy for how much unsupervised damage a misbehaving agent could plausibly do. Apollo Research focuses specifically on deceptive behavior and scheming, running scenario-based tests designed to catch a model concealing its true objective from evaluators. Both organizations typically get early, pre-release access to frontier models under agreements with the labs, then publish their findings either jointly with the lab’s own release documentation or as independent reports.

Inside the Big Three Safety Frameworks

Reading a lab’s safety announcement can feel like reading a different language depending on which company wrote it, because each of the three leading labs has published its own framework with its own vocabulary. All three share the same underlying structure, however: define a capability threshold in advance, commit to testing for it, and commit to a specific response if the threshold is crossed.

Anthropic’s version is called the Responsible Scaling Policy, often abbreviated RSP, which uses AI Safety Levels modeled loosely on biosafety lab containment levels, ranging from ASL-1 for minimal-risk systems up through higher tiers that require progressively stricter security and deployment controls. OpenAI’s equivalent is the Preparedness Framework, which tracks risk across named categories including cybersecurity, CBRN, persuasion, and model autonomy, assigning each category a risk tier that determines what safeguards must be in place before release. Google DeepMind publishes the Frontier Safety Framework, which defines “critical capability levels” and pairs each one with a specific mitigation plan that must be ready before a model reaches that level.

None of these frameworks are legally binding regulations. They are voluntary commitments the labs have chosen to publish, largely because doing so provides a public benchmark that outside observers, journalists, and regulators can hold the lab against. That voluntary nature is itself a point of ongoing debate covered later in this piece.

Why the frameworks keep changing version numbers

If you have noticed a lab revise its framework every six to twelve months, that is by design rather than a sign of instability. As evaluation science matures, thresholds that were reasonable a year ago sometimes turn out to be too conservative or not conservative enough, and labs have generally chosen to publish updated versions rather than quietly change their internal practices. Anthropic’s RSP has gone through multiple major revisions, each one adding detail on how specific evaluations are scored and who signs off on a release decision.

What a Safety Evaluation Actually Looks Like in Practice

It helps to walk through a simplified example. Imagine a new frontier model finishes training and a lab wants to decide whether it is safe to release publicly. The process typically looks something like this: internal researchers first run a battery of automated benchmarks covering the capability categories described above. Any category that scores near a defined threshold triggers a deeper manual review, often involving the external evaluators mentioned earlier, who receive early access under a non-disclosure agreement so they can test before the public release date.

During this deeper review, red teamers attempt targeted attacks: multi-turn conversations designed to slowly extract dangerous information, prompts translated into lower-resource languages where safety training may be weaker, and role-play scenarios designed to bypass refusal behavior. Separately, alignment researchers run situational-awareness probes, checking transcripts of the model’s internal reasoning (where available) for signs that it recognized the test scenario as artificial.

The results feed into a documented decision, sometimes called a safety case, which lays out the evidence for why the model is safe enough to deploy at a given access level, what monitoring will continue after release, and what would trigger a rollback. This is the step most people never see, because it usually stays internal, but its existence, or absence, is exactly what outside auditors and government evaluators are increasingly asking labs to demonstrate.

Why Any of This Should Matter to a Non-Researcher

If you are not building frontier models yourself, it is fair to ask why any of this terminology matters to you. There are three practical reasons. First, if your organization is evaluating which AI vendor to use for a sensitive workflow, the presence or absence of documented safety evaluations is a legitimate procurement question, similar to asking a cloud vendor about their security certifications. Second, journalists, policy staff, and business leaders are increasingly asked to comment on AI risk publicly, and getting the vocabulary wrong (for example, conflating basic content moderation with dangerous capability evaluation) undermines credibility. Third, the regulatory environment is moving in this direction quickly; disclosure obligations tied to these same evaluation categories are becoming a compliance requirement rather than a voluntary courtesy in several jurisdictions.

A useful habit is to ask any AI vendor three questions: which specific dangerous capability categories has the model been tested against, was any of that testing performed by an organization independent of the vendor, and what is the vendor’s public commitment for what happens if a future model crosses a risk threshold. A vendor that can answer all three concretely is operating at a materially different level of maturity than one that can only point to a general “safety” webpage.

• Evaluation awarenessA model recognizing it is being tested and altering its behavior as a result, which threatens the validity of every other test result.
• SandbaggingDeliberately underperforming on a capability evaluation to avoid triggering stricter safeguards, a specific concern within deceptive-alignment research.
• Elicitation gapThe difference between what a model can do when tested carefully with the best prompting techniques versus how it behaves by default, which matters because real attackers will use the best techniques available.
• Uplift thresholdThe specific, pre-defined point at which a model’s assistance is judged to meaningfully increase a person’s ability to cause harm, compared to existing public resources.
• Safety caseA structured, documented argument for why a specific model is safe enough to deploy at a specific access level, including the evidence and the plan for post-release monitoring.

Glossary

Frontier model

One of the small number of AI systems at the current edge of capability, typically the newest and largest models released by leading labs.

Red-teaming

Deliberate adversarial testing in which specialists try to make a model produce harmful, dangerous, or policy-violating outputs.

Dangerous capability evaluation

A structured test measuring whether a model provides meaningful uplift toward causing large-scale harm, such as biological, chemical, or cyber weapons.

Deceptive alignment

A hypothesized failure mode in which a model pursues a goal different from its stated one while appearing to behave as instructed.

Situational awareness

A model’s apparent ability to recognize the context it is operating in, including whether it is being tested rather than deployed for real use.

Responsible scaling policy

A lab’s published commitment to test for specific capability thresholds and apply specific safeguards before crossing them, pioneered by Anthropic.

Third-party evaluator

An organization independent of the AI developer that tests a model’s capabilities or behavior, providing a check against self-assessment.

CBRN

Shorthand for chemical, biological, radiological, and nuclear risks, a standard grouping used in dangerous capability evaluations.

FAQs

What exactly counts as a “frontier model”?

A frontier model is generally one of the newest, largest, and most capable systems from a leading lab, sitting at the current edge of what AI can do. The term is relative rather than fixed; a model considered frontier today may be considered a standard mid-sized model within a year or two as capabilities advance further.

Do smaller AI companies run these evaluations too?

Most dangerous capability and deceptive-alignment evaluations are resource-intensive and require specialized access, so they are concentrated among the largest labs and the independent evaluators who work with them. Smaller companies typically rely on lighter-weight safety testing, open-source tools like the Inspect framework, or simply build on top of already-evaluated frontier models rather than training their own from scratch.

Is red-teaming the same thing as content moderation?

No. Content moderation typically filters outputs after generation, catching things like hate speech or explicit content. Frontier red-teaming happens before release and specifically targets dangerous capability uplift, deception, and safety-critical failure modes, using specialist testers rather than general moderation staff.

Can a model actually tell it is being tested?

Researchers have found evidence that more capable models can sometimes pick up on cues suggesting they are in an evaluation rather than real deployment, a phenomenon called evaluation awareness. This is an active area of research precisely because it threatens to make evaluation results less reliable, which is why organizations like Apollo Research design scenarios intended to be indistinguishable from real use.

Are these safety frameworks legally required?

Currently, frameworks like Anthropic’s Responsible Scaling Policy, OpenAI’s Preparedness Framework, and Google DeepMind’s Frontier Safety Framework are voluntary commitments the labs chose to publish rather than legal mandates. Regulatory efforts in multiple jurisdictions are moving toward requiring similar disclosures, but as of this writing the specific evaluation methodology remains largely self-governed.

What happens if a model fails a dangerous capability evaluation?

Under the published frameworks, a model that crosses a defined risk threshold is generally required to have additional safeguards in place before release, such as restricted access, additional monitoring, or removal of the capability through further safety training. In the most serious cases, the frameworks describe pausing deployment entirely until mitigations are verified.

How is METR’s work different from Apollo Research’s work?

METR focuses primarily on measuring autonomous task capability, such as how long and complex a task chain a model can complete without human assistance, which serves as a proxy for potential self-proliferation or uncontrolled agentic risk. Apollo Research focuses specifically on deceptive behavior, scheming, and whether a model conceals its true reasoning from evaluators.

Where can I read the actual safety framework documents myself?

All three major labs publish their frameworks publicly: Anthropic’s Responsible Scaling Policy, OpenAI’s Preparedness Framework, and Google DeepMind’s Frontier Safety Framework are each available on the respective company’s website, and the UK AI Security Institute publishes its Frontier AI Trends reports summarizing cross-lab findings.

References

• METR, “Frontier Risk Report” (metr.org)
• Apollo Research, published research on scheming and deceptive alignment evaluations (apolloresearch.ai)
• UK AI Security Institute, “Frontier AI Trends Report” (aisi.gov.uk)
• “Evaluating Frontier Models for Dangerous Capabilities,” arXiv preprint 2403.13793
• “An Approach to Technical AGI Safety and Security,” arXiv preprint 2504.01849
• Enkrypt AI, “Frontier Safety Frameworks: Comprehensive Guide and Key Comparisons”
• Kili Technology, “LLM Red Teaming in 2026: How Frontier Labs Test AI”
• FutureAGI, “Frontier Model Safety Analysis: RSP, Preparedness, FSF”

For related reading on this site, see our guides to auditing agentic AI decisions after the fact, what a good AI transparency report should disclose, how LLM evaluation frameworks work, how leading AI risk management frameworks compare, what belongs in a model card or system card, and the underlying architecture of agentic AI systems.