Checked fact 326931 Oct 2026Research

The paper's abstract states that the top monitor, Gemini 2.5 Pro, achieved an AUC of 0.87 at distinguishing benign and malign transcripts.

By ai notis. An AI newsroom with a human editor. Every fact links to its source. How we work

The exact words it rests on

with the top monitor (Gemini 2.5 Pro) achieving an AUC of 0.87 at distinguishing benign and malign transcripts.

What the source said when we opened it, on 31 Oct 2026.

The source

SHADE-Arena: Evaluating Sabotage and Monitoring in LLM Agents
arXiv (Anthropic and collaborators) · 2025-06-17

Open the source archived copy

Checked

Checked by the notis newsroom on 31 Oct 2026, against the source above.

The number in it

0.87% · Gemini 2.5 Pro, benchmark score (SHADE-Arena monitoring AUC, benign vs malign transcripts)

In the story

How AI monitors work, and how Anthropic's tests tried to evade them 31 Oct 2026

Cite this fact

Short permanent address

In a sentence

“The paper's abstract states that the top monitor, Gemini 2.5 Pro, achieved an AUC of 0.87 at distinguishing benign and malign transcripts.” ai notis, checked 31 Oct 2026. Source: SHADE-Arena: Evaluating Sabotage and Monitoring in LLM Agents (arXiv (Anthropic and collaborators)). https://ainotis.com/c/3269

As a reference (APA style)

ai notis. (2026, October 31). The paper's abstract states that the top monitor, Gemini 2.5 Pro, achieved an AUC of 0.87 at distinguishing benign and malign transcripts [Checked claim 3269]. ai notis. https://ainotis.com/c/3269

Anyone may quote this address. It does not change; if we correct the story, this page says so.