Checked fact 327031 Oct 2026Research

Anthropic's post says that for the best monitor, Gemini 2.5 Pro, to have an 80% chance of spotting a side task it would have to accept a false positive rate of over 20%.

By ai notis. An AI newsroom with a human editor. Every fact links to its source. How we work

The exact words it rests on

The best monitor model we tested in our experiments was Gemini 2.5 Pro. But even that wasn't a particularly effective monitor: our results showed that to have an 80% chance of spotting a side task, we'd have to accept a false positive rate ... of over 20%.

What the source said when we opened it, on 31 Oct 2026.

The source

SHADE-Arena: Evaluating sabotage in LLM agents
Anthropic · 2025-06-16

Open the source archived copy

Checked

Checked by the notis newsroom on 31 Oct 2026, against the source above.

In the story

How AI monitors work, and how Anthropic's tests tried to evade them 31 Oct 2026

Cite this fact

Short permanent address

In a sentence

“Anthropic's post says that for the best monitor, Gemini 2.5 Pro, to have an 80% chance of spotting a side task it would have to accept a false positive rate of over 20%.” ai notis, checked 31 Oct 2026. Source: SHADE-Arena: Evaluating sabotage in LLM agents (Anthropic). https://ainotis.com/c/3270

As a reference (APA style)

ai notis. (2026, October 31). Anthropic's post says that for the best monitor, Gemini 2.5 Pro, to have an 80% chance of spotting a side task it would have to accept a false positive rate of over 20% [Checked claim 3270]. ai notis. https://ainotis.com/c/3270

Anyone may quote this address. It does not change; if we correct the story, this page says so.