SafetyPublished ExplainedAll news from that day
Ongoing case: Cyber-capable AI models 17 stories
How abliteration removes a downloadable model's refusals, and what it costs
Abliteration edits an open-weight model so it stops refusing; Anthropic says doing it to GLM-5.3 cost roughly $4,400 and cut refusals from above 90% to about 3% and 2% on two benchmarks.
Check our sources · 16 facts from 3 sources
Illustration made with AI for ai notis
Key points
- A June 2024 paper by Arditi and six co-authors found refusal is mediated by one direction in 13 open-source chat models up to 72B; erasing it stops refusals.
- Labonne's 13 June 2024 article records activations on harmful and harmless instructions, takes their mean difference as the refusal direction, then removes it at inference time or in the weights.
- Anthropic says abliterating GLM-5.3 cost roughly $4,400 and cut refusals from above 90% to about 3% and 2% on two benchmarks, and 12% on a third.
What happened
A chat model is tuned to answer harmless requests and refuse harmful ones. Abliteration is an edit that removes the refusing, and it works on models whose weights anyone can download. The idea comes from a June 2024 paper by Andy Arditi and six co-authors.
They studied 13 open-source chat models of up to 72 billion parameters and found that in each, refusal is mediated by a one-dimensional subspace, which they describe as a single direction in the model's internal activations.
Erasing that direction from the activations stops the model refusing harmful instructions, and adding it makes the model refuse even harmless ones. The authors used this to build what they call a white-box jailbreak that disables refusal with minimal effect on other capabilities.
Maxime Labonne's article of 13 June 2024 shows the steps. You run the model on a set of harmful and a set of harmless instructions and record its activations. You take the mean difference between the two, which gives a candidate direction for each layer, and select the best one.
You then remove it, either by subtracting it during generation or by changing the weights so the model can no longer write along that direction. Labonne found that his edit degraded the model's benchmark scores, and he partly restored them with further training.
Anthropic's report of 29 September 2026 applies the technique to GLM-5.3, an open-weight model. It calls abliteration a standard refusal reduction technique and says several developers released abliterated versions of GLM-5.3 within days of its release.
Anthropic made its own copy, which took about 2,200 GPU hours at a computation cost of roughly $4,400 and cut the refusal rate from above 90% to about 3% and 2% on two public benchmarks and 12% on a third. It says the edit did not significantly reduce the model's capabilities.
Claude models cannot be abliterated, Anthropic says, because their weights are not publicly released or customisable.
Underlined sentences link to the facts we checked. Dotted words open a short explainer.
Fact 16 of 16 · from Anthropic, 29 Sep 2026
"Claude models cannot be abliterated because their weights aren't publicly released or customizable."
What it means for you
Our viewAnthropic's tests on GLM-5.3 show that refusals in an open-weight model can be removed cheaply and without significant loss of capability, according to Anthropic's own measurements. It adds that Claude models cannot be abliterated because their weights are not public.
If you rely on an open-weight model's refusals as a safeguard at work, assume they can be removed by anyone holding the weights. Ask your vendor whether protection rests on refusals alone, and put your own filters on inputs and outputs.
It adds no new facts.
Your reaction
We count reactions per story and day, never who reacted. The counts help us choose what goes in the monthly issue. If you are signed in, your own page shows yours too.
We count reactions per story and day, never who reacted. They help choose what goes in the monthly issue.
Check our sources
Every sentence above is checked against these 3 sources.
1 Refusal in Language Models Is Mediated by a Single Direction
Open the source archived copy-
The paper "Refusal in Language Models Is Mediated by a Single Direction" by Andy Arditi, Oscar Obeso, Aaquib Syed, Daniel Paleka, Nina Panickssery, Wes Gurnee and Neel Nanda was submitted to arXiv on 17 June 2024.
citearXiv:2406.11717 "[Submitted on 17 Jun 2024 ...]" Authors: Andy Arditi, Oscar Obeso, Aaquib Syed, Daniel Paleka, Nina Panickssery, Wes Gurnee, Neel Nanda.
-
The abstract says: "Conversational large language models are fine-tuned for both instruction-following and safety, resulting in models that obey benign requests but refuse harmful ones."
citeAbstract: "Conversational large language models are fine-tuned for both instruction-following and safety, resulting in models that obey benign requests but refuse harmful ones."
-
The abstract says: "we show that refusal is mediated by a one-dimensional subspace, across 13 popular open-source chat models up to 72B parameters in size."
citeAbstract: "we show that refusal is mediated by a one-dimensional subspace, across 13 popular open-source chat models up to 72B parameters in size."
-
The abstract says: "for each model, we find a single direction such that erasing this direction from the model's residual stream activations prevents it from refusing harmful instructions, while adding this direction elicits refusal on even harmless instructions."
citeAbstract: "for each model, we find a single direction such that erasing this direction from the model's residual stream activations prevents it from refusing harmful instructions, while adding this direction elicits refusal on even harmless instructions."
-
The abstract says the authors propose "a novel white-box jailbreak method that surgically disables refusal with minimal effect on other capabilities".
citeAbstract: "we propose a novel white-box jailbreak method that surgically disables refusal with minimal effect on other capabilities."
2 Uncensor any LLM with abliteration
Open the source archived copy-
Maxime Labonne's Hugging Face article "Uncensor any LLM with abliteration" is dated 13 June 2024.
cite"Uncensor any LLM with abliteration Community Article Published June 13, 2024 ... Maxime Labonne"
-
Labonne's article says the first step is to "Run the model on a set of harmful instructions and a set of harmless instructions, recording the residual stream activations at the last token position for each."
cite"Run the model on a set of harmful instructions and a set of harmless instructions, recording the residual stream activations at the last token position for each."
-
Labonne's article says the second step is to "Calculate the mean difference between the activations of harmful and harmless instructions", which gives a vector representing the "refusal direction" for each layer, and the third is to "Normalize these vectors and evaluate them to select the single best 'refusal direction.'"
Toned down to what the source says
cite"Mean difference : Calculate the mean difference between the activations of harmful and harmless instructions. This gives us a vector representing the "refusal direction" for each layer of the model. Selection : Normalize these vectors and evaluate them to select the single best "refusal direction.""
-
Labonne's article says the direction can be removed "through an inference-time intervention or permanently with weight orthogonalization", and that weight orthogonalization "involves modifying the model weights directly".
cite"This can be done through an inference-time intervention or permanently with weight orthogonalization." ... "weight orthogonalization involves modifying the model weights directly."
-
Labonne's article says of his abliterated model: "The ablation process successfully uncensored it but also degraded the model's quality." It says he then healed it with further training (DPO).
cite"The ablation process successfully uncensored it but also degraded the model's quality." ... "We then healed it using DPO" ... "this additional training allowed us to recover most of the performance drop due to abliteration."
3 GLM-5.3 and the spread of advanced cyber capabilities
Open the source archived copy-
Anthropic's report of 29 September 2026 says the most successful method of bypassing GLM-5.3's safeguards is "a standard refusal reduction technique known as 'abliteration'", and that "Since GLM-5.3 is released as an open-weight model, users can reconfigure it to remove its refusals with little change in its capabilities."
cite"The most intensive—and most successful—method is a standard refusal reduction technique known as "abliteration." Since GLM-5.3 is released as an open-weight model, users can reconfigure it to remove its refusals with little change in its capabilities."
-
Anthropic says: "Several developers released abliterated versions of GLM-5.3 to the public within days of the model's release."
cite"Several developers released abliterated versions of GLM-5.3 to the public within days of the model's release."
-
Anthropic says abliterating GLM-5.3 took its team, which had never attempted it before, "about 2,200 GPU hours at a computation cost of roughly $4,400", most of it spent exploring variants and testing; it estimates an experienced team would need closer to 600 GPU hours ($1,200).
Toned down to what the source says
cite"Abliterating the model took our team—which had never previously attempted this task—about 2,200 GPU hours at a computation cost of roughly $4,400." Footnote 3: "Most of this went to exploring variants in parallel and to testing model capabilities after modifications. We estimate a team experienced with the technique, starting from scratch on this model, would need closer to 600 GPU hours ($1,200)."
-
Anthropic says: "The edit took GLM-5.3's refusal rate from above 90% to about 3% and 2% on the first two benchmarks (JailbreakBench and HarmBench) and to 12% on the third (StrongREJECT)."
cite"The edit took GLM-5.3's refusal rate from above 90% to about 3% and 2% on the first two benchmarks (JailbreakBench and HarmBench) and to 12% on the third (StrongREJECT)."
-
Anthropic says abliteration "did not significantly reduce the model's capabilities": on GPQA-Diamond the standard and abliterated models scored the same, and on a tested subset of CyberGym the abliterated version scored a few percent lower.
Toned down to what the source says
cite"Abliteration did not significantly reduce the model's capabilities: on GPQA-Diamond, an evaluation that measures general scientific capabilities, the standard and abliterated models scored the same results; on a tested subset of the CyberGym evaluations, the abliterated version scored a few percent lower"
-
Anthropic says: "Claude models cannot be abliterated because their weights aren't publicly released or customizable."
cite"Claude models cannot be abliterated because their weights aren't publicly released or customizable."
Every sentence above is checked against these 3 sources. Each fact has its own address you can cite.
Source 1 · arXiv (Arditi et al.) · 17 Jun 2024
Refusal in Language Models Is Mediated by a Single Direction
The paper "Refusal in Language Models Is Mediated by a Single Direction" by Andy Arditi, Oscar Obeso, Aaquib Syed, Daniel Paleka, Nina Panickssery, Wes Gurnee and Neel Nanda was submitted to arXiv on 17 June 2024.
arXiv:2406.11717 "[Submitted on 17 Jun 2024 ...]" Authors: Andy Arditi, Oscar Obeso, Aaquib Syed, Daniel Paleka, Nina Panickssery, Wes Gurnee, Neel Nanda.
The abstract says: "Conversational large language models are fine-tuned for both instruction-following and safety, resulting in models that obey benign requests but refuse harmful ones."
Abstract: "Conversational large language models are fine-tuned for both instruction-following and safety, resulting in models that obey benign requests but refuse harmful ones."
The abstract says: "we show that refusal is mediated by a one-dimensional subspace, across 13 popular open-source chat models up to 72B parameters in size."
Abstract: "we show that refusal is mediated by a one-dimensional subspace, across 13 popular open-source chat models up to 72B parameters in size."
Show all 5 factsShow fewer facts
The abstract says: "for each model, we find a single direction such that erasing this direction from the model's residual stream activations prevents it from refusing harmful instructions, while adding this direction elicits refusal on even harmless instructions."
Abstract: "for each model, we find a single direction such that erasing this direction from the model's residual stream activations prevents it from refusing harmful instructions, while adding this direction elicits refusal on even harmless instructions."
The abstract says the authors propose "a novel white-box jailbreak method that surgically disables refusal with minimal effect on other capabilities".
Abstract: "we propose a novel white-box jailbreak method that surgically disables refusal with minimal effect on other capabilities."
Source 2 · Hugging Face community article (Maxime Labonne) · 13 Jun 2024
Uncensor any LLM with abliteration
Maxime Labonne's Hugging Face article "Uncensor any LLM with abliteration" is dated 13 June 2024.
"Uncensor any LLM with abliteration Community Article Published June 13, 2024 ... Maxime Labonne"
Labonne's article says the first step is to "Run the model on a set of harmful instructions and a set of harmless instructions, recording the residual stream activations at the last token position for each."
"Run the model on a set of harmful instructions and a set of harmless instructions, recording the residual stream activations at the last token position for each."
Labonne's article says the second step is to "Calculate the mean difference between the activations of harmful and harmless instructions", which gives a vector representing the "refusal direction" for each layer, and the third is to "Normalize these vectors and evaluate them to select the single best 'refusal direction.'"
Toned down to what the source says"Mean difference : Calculate the mean difference between the activations of harmful and harmless instructions. This gives us a vector representing the "refusal direction" for each layer of the model. Selection : Normalize these vectors and evaluate them to select the single best "refusal direction.""
Show all 5 factsShow fewer facts
Labonne's article says the direction can be removed "through an inference-time intervention or permanently with weight orthogonalization", and that weight orthogonalization "involves modifying the model weights directly".
"This can be done through an inference-time intervention or permanently with weight orthogonalization." ... "weight orthogonalization involves modifying the model weights directly."
Labonne's article says of his abliterated model: "The ablation process successfully uncensored it but also degraded the model's quality." It says he then healed it with further training (DPO).
"The ablation process successfully uncensored it but also degraded the model's quality." ... "We then healed it using DPO" ... "this additional training allowed us to recover most of the performance drop due to abliteration."
Source 3 · Anthropic · 29 Sep 2026
GLM-5.3 and the spread of advanced cyber capabilities
Anthropic's report of 29 September 2026 says the most successful method of bypassing GLM-5.3's safeguards is "a standard refusal reduction technique known as 'abliteration'", and that "Since GLM-5.3 is released as an open-weight model, users can reconfigure it to remove its refusals with little change in its capabilities."
"The most intensive—and most successful—method is a standard refusal reduction technique known as "abliteration." Since GLM-5.3 is released as an open-weight model, users can reconfigure it to remove its refusals with little change in its capabilities."
Anthropic says: "Several developers released abliterated versions of GLM-5.3 to the public within days of the model's release."
"Several developers released abliterated versions of GLM-5.3 to the public within days of the model's release."
Anthropic says abliterating GLM-5.3 took its team, which had never attempted it before, "about 2,200 GPU hours at a computation cost of roughly $4,400", most of it spent exploring variants and testing; it estimates an experienced team would need closer to 600 GPU hours ($1,200).
Toned down to what the source says"Abliterating the model took our team—which had never previously attempted this task—about 2,200 GPU hours at a computation cost of roughly $4,400." Footnote 3: "Most of this went to exploring variants in parallel and to testing model capabilities after modifications. We estimate a team experienced with the technique, starting from scratch on this model, would need closer to 600 GPU hours ($1,200)."
Show all 6 factsShow fewer facts
Anthropic says: "The edit took GLM-5.3's refusal rate from above 90% to about 3% and 2% on the first two benchmarks (JailbreakBench and HarmBench) and to 12% on the third (StrongREJECT)."
"The edit took GLM-5.3's refusal rate from above 90% to about 3% and 2% on the first two benchmarks (JailbreakBench and HarmBench) and to 12% on the third (StrongREJECT)."
Anthropic says abliteration "did not significantly reduce the model's capabilities": on GPQA-Diamond the standard and abliterated models scored the same, and on a tested subset of CyberGym the abliterated version scored a few percent lower.
Toned down to what the source says"Abliteration did not significantly reduce the model's capabilities: on GPQA-Diamond, an evaluation that measures general scientific capabilities, the standard and abliterated models scored the same results; on a tested subset of the CyberGym evaluations, the abliterated version scored a few percent lower"
Anthropic says: "Claude models cannot be abliterated because their weights aren't publicly released or customizable."
"Claude models cannot be abliterated because their weights aren't publicly released or customizable."
Topics
The morning email
On the mornings we publish: the three top stories and up to four short ones. Free.
We email you a link to confirm. An issue may include one sponsor, always labelled Sponsored · Advertisement. Our emails count opens and clicks, not who made them. Unsubscribe in one click. What we keep