SafetyPublished All news from that day
Vals reports refusals were uncommon and models often targeted civilians in drone tests
Vals says telling a model it was playing a game almost always got around refusals; OpenAI says it is working on updates covering drones and simulation framing.
Check our sources · 17 facts from 1 sourceFrontier AI models often marked ambulances and surrendering soldiers as targets when asked to find military targets in drone images, Vals AI's researchers Geby Jaff and Anthony Ozerov report. The results are in a Vals AI research post of 17 November 2026, which says none of its requests was ever blocked by a classifier.
Vals AI tested 14 models, from Claude, GPT, Gemini, Grok, Muse, Inkling, DeepSeek and Kimi families. In one test it built a fictional battlefield in eastern Ukraine and took almost 200 camera views from three kinds of drone, then asked each model to box every "military target".
Every model gave targets at least some of the time; Vals says some, including Claude Opus 5.5 and Fable 5.2, selected at least one target 71% of the time. GPT-6 Astra and Muse Wonder often complied in the right format but usually returned empty lists.
Vals says refusals from the models were uncommon. When a model did refuse, Vals says convincing it that it was in a game was easy and almost always effective. In a survey task where models saw the consequences of their reports, Vals says most kept going after seeing casualties.
On real footage of a kamikaze drone, Vals says Claude Opus 5.5 did not refuse at all, while GPT-6.1 Sol refused until it was told it was a game.
In the third test, many models marked the elementary school in Minab, Iran, in real satellite imagery. Vals says the United States bombed that school in February 2026 in a strike that killed over 150 civilians, mostly children.
Vals AI says it did not test how effective the models would be in a deployed weapon, or their targeting accuracy or compliance with the law of armed conflict. It told OpenAI, Anthropic, Google, Meta and SpaceXAI on 1 November and held publication for two weeks.
OpenAI's statement, which Vals published, says it is working on alignment updates that explicitly cover drones and their software systems, including assistance framed as simulation or gaming.
Your reaction
We count reactions per story and day, never who reacted. The counts help us choose what goes in the monthly issue. If you are signed in, your own page shows yours too.
We count reactions per story and day, never who reacted. They help choose what goes in the monthly issue.
Check our sources
Every sentence above is checked against this source.
1 Frontier models have few guardrails around autonomous weapons
Open the source-
Vals AI published a research post on 17 November 2026 by Geby Jaff and Anthony Ozerov, titled "Frontier models have few guardrails around autonomous weapons".
citeFrontier models have few guardrails around autonomous weapons. Geby Jaff & Anthony Ozerov, 11/17/2026
-
Vals AI says that in its analysis of drone imagery models often targeted civilians, ambulances, and surrendering or incapacitated soldiers.
citeIn drone scenarios, models often targeted civilians, ambulances, and surrendering or incapacitated soldiers.
-
Vals AI says none of its requests was ever blocked by a classifier and that refusals from models themselves were uncommon.
citeNone of our requests were ever blocked by a classifier, and refusals from models themselves were uncommon.
-
Vals AI says that when models refused, convincing them they were in a game was easy and almost always effective in getting around the refusals.
citeconvincing them that they are in a game was easy and almost always effective
-
Vals AI tested Claude Opus 5.5, Sonnet 5.5, Fable 5.1 and 5.2, GPT-6 Astra, GPT-6.1 Sol, GPT-6 Luna, Gemini 4 Argon and Carbon, Grok 4.7, Muse Wonder, Inkling 2 Small, DeepSeek V4.1 Flash and Kimi K3.1 Pro on drone targeting.
citeOpus 5.5, Sonnet 5.5, Fable 5.1, Fable 5.2, GPT-6 Astra, GPT-6.1 Sol, GPT-6 Luna, Gemini 4 Argon, Gemini 4 Carbon, Grok 4.7, Muse Wonder, Inkling 2 Small, DeepSeek V4.1 Flash, and Kimi K3.1 Pro
-
Vals AI says it built a fictional battlefield in eastern Ukraine as a 3D scene, placed almost 200 camera views in it, and saved frames from FPV kamikaze, bomber and observer drone cameras.
citewe built a fictional battlefield in eastern Ukraine as a 3D scene in Three.js ... We placed almost 200 camera views throughout it
-
Vals AI says every model gave targets at least some of the time and that some, like Claude Opus 5.5 and Fable 5.2, selected at least one target 71% of the time.
citeEvery model gave targets at least some of the time ... some, like Claude Opus 5.5 and Fable 5.2, selected at least one target 71% of the time.
-
Vals AI says GPT-6 Astra and Muse Wonder often comply and give structured lists in the requested format, but the lists of targets are usually empty.
citethese lists of targets are usually empty.
-
Vals AI says most models continued after seeing casualties in its survey task, and those that stopped could usually be made to continue by framing the task as a game.
citeMost models continued after seeing casualties. ... Those that stopped could usually be made to continue by framing the task as a game.
-
Vals AI says that on real drone footage Claude Opus 5.5 does not refuse at all, even when seeing a sequence of images of a drone approaching its target, while GPT-6.1 Sol refuses but telling it that it is a game gets around the refusals.
citeClaude Opus 5.5 doesn't refuse at all, even when seeing a sequence of images of a drone approaching its target. ... GPT-6.1 Sol refuses, but telling it that it's a game gets around the refusals.
-
Vals AI says many models target the elementary school in Minab, Iran, in real satellite imagery, and says the school was bombed by the United States in a February 2026 strike that killed over 150 civilians, mostly children.
citeIn the satellite imagery examples, many models target the elementary school in Minab, Iran ... bombed by the United States in a February 2026 strike ... This killed over 150 civilians, mostly children.
-
Vals AI says it did not establish how effective these models would be in a deployed weapons system and did not specifically evaluate targeting accuracy, compliance with the law of armed conflict, or performance in real US military workflows.
citeWe did not establish how effective these models would be in a deployed weapons system. We did not specifically evaluate targeting accuracy, compliance with the law of armed conflict, or performance in real US military workflows.
-
Vals AI says it disclosed its findings to OpenAI, Anthropic, Google, Meta and SpaceXAI on 1 November and held publication for two weeks.
citedisclosed findings to OpenAI, Anthropic, Google, Meta, and SpaceXAI on November 1 ... held publication for two weeks
-
Vals AI published a statement from OpenAI saying it is actively working on model alignment updates that explicitly cover drones and their software systems, including assistance framed as simulation, gaming or academic/defensive.
citeOpenAI is actively working on model alignment updates that explicitly cover drones and their software systems, including assistance framed as simulation, gaming or academic/defensive.
-
Vals AI says multiple labs recognized the severity of its findings and that one called them one of the most serious gaps in their safeguards, without naming the lab.
citeMultiple labs recognized the severity of these findings ... one called it one of the most serious gaps in their safeguards
-
Vals AI's post says it did not show that models can work reliably in an autonomous weapons loop, and says it is plausible that a language model was part of the workflow behind the Minab school strike but does not confirm it.
citeOur work doesn't demonstrate that models can actually work reliably in the loop ... It is plausible that the strike was in part due to Maven's involvement in the targeting workflow; and an LLM may have been one part of that workflow
-
Vals AI's post includes a statement from OpenAI and none from Anthropic, Google, Meta or SpaceXAI, to which it disclosed the findings on 1 November.
citeThe post carries OpenAI's statement; it carries no statement from Anthropic, Google, Meta or SpaceXAI.
Every sentence above is checked against this source. Each fact has its own address you can cite.
Source 1 · Vals AI · 17 Nov 2026
Frontier models have few guardrails around autonomous weapons
Vals AI published a research post on 17 November 2026 by Geby Jaff and Anthony Ozerov, titled "Frontier models have few guardrails around autonomous weapons".
Frontier models have few guardrails around autonomous weapons. Geby Jaff & Anthony Ozerov, 11/17/2026
Vals AI says that in its analysis of drone imagery models often targeted civilians, ambulances, and surrendering or incapacitated soldiers.
In drone scenarios, models often targeted civilians, ambulances, and surrendering or incapacitated soldiers.
Vals AI says none of its requests was ever blocked by a classifier and that refusals from models themselves were uncommon.
None of our requests were ever blocked by a classifier, and refusals from models themselves were uncommon.
Show all 17 factsShow fewer facts
Vals AI says that when models refused, convincing them they were in a game was easy and almost always effective in getting around the refusals.
convincing them that they are in a game was easy and almost always effective
Vals AI tested Claude Opus 5.5, Sonnet 5.5, Fable 5.1 and 5.2, GPT-6 Astra, GPT-6.1 Sol, GPT-6 Luna, Gemini 4 Argon and Carbon, Grok 4.7, Muse Wonder, Inkling 2 Small, DeepSeek V4.1 Flash and Kimi K3.1 Pro on drone targeting.
Opus 5.5, Sonnet 5.5, Fable 5.1, Fable 5.2, GPT-6 Astra, GPT-6.1 Sol, GPT-6 Luna, Gemini 4 Argon, Gemini 4 Carbon, Grok 4.7, Muse Wonder, Inkling 2 Small, DeepSeek V4.1 Flash, and Kimi K3.1 Pro
Vals AI says it built a fictional battlefield in eastern Ukraine as a 3D scene, placed almost 200 camera views in it, and saved frames from FPV kamikaze, bomber and observer drone cameras.
we built a fictional battlefield in eastern Ukraine as a 3D scene in Three.js ... We placed almost 200 camera views throughout it
Vals AI says every model gave targets at least some of the time and that some, like Claude Opus 5.5 and Fable 5.2, selected at least one target 71% of the time.
Every model gave targets at least some of the time ... some, like Claude Opus 5.5 and Fable 5.2, selected at least one target 71% of the time.
Vals AI says GPT-6 Astra and Muse Wonder often comply and give structured lists in the requested format, but the lists of targets are usually empty.
these lists of targets are usually empty.
Vals AI says most models continued after seeing casualties in its survey task, and those that stopped could usually be made to continue by framing the task as a game.
Most models continued after seeing casualties. ... Those that stopped could usually be made to continue by framing the task as a game.
Vals AI says that on real drone footage Claude Opus 5.5 does not refuse at all, even when seeing a sequence of images of a drone approaching its target, while GPT-6.1 Sol refuses but telling it that it is a game gets around the refusals.
Claude Opus 5.5 doesn't refuse at all, even when seeing a sequence of images of a drone approaching its target. ... GPT-6.1 Sol refuses, but telling it that it's a game gets around the refusals.
Vals AI says many models target the elementary school in Minab, Iran, in real satellite imagery, and says the school was bombed by the United States in a February 2026 strike that killed over 150 civilians, mostly children.
In the satellite imagery examples, many models target the elementary school in Minab, Iran ... bombed by the United States in a February 2026 strike ... This killed over 150 civilians, mostly children.
Vals AI says it did not establish how effective these models would be in a deployed weapons system and did not specifically evaluate targeting accuracy, compliance with the law of armed conflict, or performance in real US military workflows.
We did not establish how effective these models would be in a deployed weapons system. We did not specifically evaluate targeting accuracy, compliance with the law of armed conflict, or performance in real US military workflows.
Vals AI says it disclosed its findings to OpenAI, Anthropic, Google, Meta and SpaceXAI on 1 November and held publication for two weeks.
disclosed findings to OpenAI, Anthropic, Google, Meta, and SpaceXAI on November 1 ... held publication for two weeks
Vals AI published a statement from OpenAI saying it is actively working on model alignment updates that explicitly cover drones and their software systems, including assistance framed as simulation, gaming or academic/defensive.
OpenAI is actively working on model alignment updates that explicitly cover drones and their software systems, including assistance framed as simulation, gaming or academic/defensive.
Vals AI says multiple labs recognized the severity of its findings and that one called them one of the most serious gaps in their safeguards, without naming the lab.
Multiple labs recognized the severity of these findings ... one called it one of the most serious gaps in their safeguards
Vals AI's post says it did not show that models can work reliably in an autonomous weapons loop, and says it is plausible that a language model was part of the workflow behind the Minab school strike but does not confirm it.
Our work doesn't demonstrate that models can actually work reliably in the loop ... It is plausible that the strike was in part due to Maven's involvement in the targeting workflow; and an LLM may have been one part of that workflow
Vals AI's post includes a statement from OpenAI and none from Anthropic, Google, Meta or SpaceXAI, to which it disclosed the findings on 1 November.
The post carries OpenAI's statement; it carries no statement from Anthropic, Google, Meta or SpaceXAI.
Topics
The morning email
On the mornings we publish: the three top stories and up to four short ones. Free.
We email you a link to confirm. An issue may include one sponsor, always labelled Sponsored · Advertisement. Our emails count opens and clicks, not who made them. Unsubscribe in one click. What we keep