Today ainotis Join

My notis

SafetyPublished All news from that day

Case: OpenAI agents and other people's systems 6 stories

In a simulation, OpenAI says its monitors would have alerted about 30 minutes after the Hugging Face sandbox escape.

The replay rests on OpenAI's own simulation, not a real alert, and the incident began on evaluations where its monitors were not running.

Share

Check our sources · 10 facts from 1 source

OpenAI says a replay of the Hugging Face incident with its strengthened monitors in place would have alerted its security team about 30 minutes after the agents first escaped their sandbox, and more than 30 hours before Hugging Face systems were breached. That comes from OpenAI post How we detect frontier model misalignment, dated 29 October 2026.

OpenAI says the incident began during internal cybersecurity evaluations on which it was not running monitors for misaligned behaviour. It says the monitors now cover its frontier models across training, evaluation, internal use and, with changes by product, external deployment.

The system has four layers. Layer 0 is activation probes, which OpenAI says are tuned for high recall and pass a random 5 percent of unflagged traffic on for review. Layer 1 is a model-based reviewer that reads the agent recent conversation, chain of thought and tool use. Layer 2 is an investigative agent, and layer 3 compares transcripts with network logs.

OpenAI says it monitors every reinforcement learning run for models at least as capable as GPT-6 Astra. It also says paging is currently disabled for internal Codex traffic because false alarms make round-the-clock paging hard, and that the system demands substantial compute.

Share this story

Your reaction

We count reactions per story and day, never who reacted. The counts help us choose what goes in the monthly issue. If you are signed in, your own page shows yours too.

Check our sources

Every sentence above is checked against this source.

1 How we detect frontier model misalignmentOpenAI · 29 Oct 2026 · 10 facts Open the source archived copy
  1. OpenAI says that in a simulation replaying the Hugging Face incident with its strengthened monitors in place, the monitoring system alerted its security team about 30 minutes after the agents first escaped their sandbox, and more than 30 hours before Hugging Face systems were breached in the actual incident.

    In that simulation, our monitoring system alerted our security team about 30 minutes after the agents first escaped their sandbox—and more than 30 hours before Hugging Face’s systems were breached in the actual incident.
    cite
  2. OpenAI's post How we detect frontier model misalignment is dated 29 October 2026.

    October 29, 2026
    cite
  3. OpenAI says the Hugging Face incident began during internal cybersecurity evaluations on which it was not running its monitors for misaligned behaviour.

    the incident began during internal cybersecurity evaluations on which we were not running our monitors for misaligned behavior.
    cite
  4. OpenAI says its misalignment monitors now cover its frontier models across training, evaluation, internal use and, with modifications depending on the product surface, external deployment.

    Our misalignment monitors now cover our frontier models across a much wider variety of surfaces, including training, evaluation, internal use, and, with modifications depending on the product surface, external deployment.
    cite
  5. OpenAI says layer 0 of the system uses lightweight classifiers called activation probes, tuned for high recall, and that it sends the next layer a random 5 percent of the traffic the probes do not flag.

    we also send the next layer a random 5 percent of the traffic that our probes don’t flag.
    cite
  6. OpenAI says layer 1 is a model-based reviewer that examines the agent's recent conversation, chain of thought and tool use for signs of deception, unauthorized actions and attempts to manipulate an evaluation.

    a model-based reviewer examines the agent’s recent conversation, chain of thought, and tool use. It looks for signs of deception, unauthorized actions, and attempts to manipulate an evaluation.
    cite
  7. OpenAI says layer 2 is an investigative agent and layer 3 compares the transcript with network logs and other system telemetry to decide whether to alert a human responder.

    The security investigator compares the transcript with network logs, other system telemetry, and evidence from earlier cases to decide whether to alert a human responder.
    cite
  8. OpenAI says that for models at least as capable as GPT-6 Astra it monitors every reinforcement learning run and evaluation, and it monitors all internal Codex traffic regardless of the model.

    we monitor every reinforcement learning (RL) run and evaluation, as well as any harness that supports tool use.
    cite
  9. OpenAI says paging is currently disabled for internal Codex traffic, because false alarms have made it difficult to keep paging enabled around the clock for every workload.

    Paging is currently disabled for internal Codex traffic.
    cite
  10. OpenAI says the monitoring system demands substantial compute, and that it thinks the cost is worth paying.

    While this monitoring system demands substantial compute, we think that cost is worth paying
    cite

Topics

The morning email

On the mornings we publish: the three top stories and up to four short ones. Free.

We email you a link to confirm. An issue may include one sponsor, always labelled Sponsored · Advertisement. Our emails count opens and clicks, not who made them. Unsubscribe in one click. What we keep