SafetyPublished Top storyAll news from that day
High risk of an AI takeover within three years, Paul Christiano says
In the Hinton Lectures, the alignment researcher said the fastest build of AI means something like a coin flip of an AI takeover, and development should probably slow down.
Check our sources · 15 facts from 6 sources
Video: The Hinton Lectures™ on YouTube
Key points
- Christiano says: "I think there is a high risk of an AI takeover within the next three years." (14:31)
- On building AI "as quickly as we can", he says "we would be looking at something like a coin flip of an AI takeover". (15:50)
- In the third lecture he said "we should probably slow down immediately until we actually have the infrastructure to slow down later". (41:43)
What happened
Paul Christiano, an AI alignment researcher, says there is "a high risk of an AI takeover within the next three years". He said so in the first of three Hinton Lectures in Toronto on 9 November, a series hosted by Geoffrey Hinton and put on by the AI Safety Foundation.
By takeover he means "AI systems pursuing goals, and those systems call the shots in the same sense that humans today call the shots". He puts no figure on the risk: he says "any number we give is an expression of someone's subjective opinion".
If AI were built along the most rapid trajectory, he said, "we would be looking at something like a coin flip of an AI takeover".
He went on: "I hope the probability is much lower than that because we are not going to develop the technology in the very fastest way we can, but I think even all things considered, with realistic policy responses, the risk remains exceptionally high."
Christiano also gave shorter horizons. In the first lecture he said there is "meaningful risk even in the next 6 to 12 months", and in the second he spoke of "material risk even within the next six to 18 months".
On the third night, on 11 November, he said what he thinks should follow: "we should probably slow down immediately until we actually have the infrastructure to slow down later". He tied that to having "security, governance, risk management" good enough "to detect and react to future risks".
Christiano is the executive director of the Alignment Research Center, which he founded, and a Senior Tech Advisor at the US Center for AI Standards and Innovation. On 9 September 2026 OpenAI announced his appointment to the board of the OpenAI Foundation and its Safety and Security Committee.
What it means for you
Our viewThis is one researcher's view, and Christiano says himself that any number given for this risk is someone's subjective opinion, and that he does not think a quantitative model exists to put a number on it.
What he asks for is conditional: slowing down until security, governance and risk management are good enough to detect and react to future risks as they emerge. He also sits on the OpenAI Foundation board and its Safety and Security Committee, so his lectures are worth reading with that role in mind.
If you set or review AI policy at work, list which conditions in your own safety framework would make you pause a rollout, and write down who decides.
It adds no new facts.
Your reaction
We count reactions per story and day, never who reacted. The counts help us choose what goes in the monthly issue. If you are signed in, your own page shows yours too.
We count reactions per story and day, never who reacted. They help choose what goes in the monthly issue.
Check our sources
Every sentence above is checked against these 6 sources.
1 Paul Christiano: AI Alignment - What's Working and What's Not | The Hinton Lectures 2026 (Night 1)
Open the source archived copy-
Paul Christiano said at the first of the Hinton Lectures: "I think there is a high risk of an AI takeover within the next three years." (14:31)
citeUh so I think there is a high risk of an AI takeover within the next three years.
-
Christiano described an AI takeover as a situation where there are "AI systems pursuing goals, and those systems call the shots in the same sense that humans today call the shots" (14:37)
citea system situation where there are AI systems pursuing goals, and those systems call the shots in the same sense that humans today call the shots and what happens on Earth.
-
Christiano said: "I don't think we have any kind of quantitative model we can turn to to put a number on this event." (15:22)
citeI don't think we have any kind of quantitative model we can turn to to put a number on this event.
-
Christiano said: "Any number we give is an expression of someone's subjective opinion." (15:35)
citeAny number we give is an expression of someone's subjective opinion.
-
Christiano said: "if we tried to develop AI along the most rapid trajectory, if we said we're just going to build this as quickly as we can in the easiest way we can, we would be looking at something like a coin flip of an AI takeover." (15:50)
citeif we tried to develop AI along the most rapid trajectory, if we said we're just going to build this as quickly as we can in the easiest way we can, we would be looking at something like a coin flip of an AI takeover.
-
Christiano said: "I hope the probability is much lower than that because we are not going to develop the technology in the very fastest way we can, but I think even all things considered, with realistic policy responses, the risk remains exceptionally high." (16:02)
Toned down to what the source says
citeI hope the probability is much lower than that because we are not going to develop the technology in the very fastest way we can, but I think even all things considered, with realistic policy responses, the risk remains exceptionally high.
-
Christiano said: "I think there's meaningful risk even in the next 6 to 12 months, right?" (16:31)
citeI think there's meaningful risk even in the next 6 to 12 months, right?
2 Paul Christiano: Staying in Control of Self-Improving AI | The Hinton Lectures 2026 (Night 2)
Open the source archived copy-
In the second lecture Christiano said he would talk about a shorter time scale and "why I think there is material risk even within the next six to 18 months." (12:09)
cite
3 Paul Christiano: Governing Transformative AI | The Hinton Lectures 2026 (Night 3)
Open the source archived copy-
In the third lecture Christiano said: "my view is now we should probably slow down immediately until we actually have the infrastructure to slow down later" (41:43)
Toned down to what the source says
citeSo in light of that, my view is now we should probably slow down immediately until we actually have the infrastructure to slow down later,
-
Christiano said slowing down should last "until we have a high enough degree of security, governance, risk management that we are able to detect and react to future risks as they emerge." (41:50)
citeUntil we have a high enough degree of security, governance, risk management that we are able to detect and react to future risks as they emerge.
4 The Hinton Lectures 2026
Open the source archived copy-
The Hinton Lectures 2026 ran from 9 to 11 November 2026 at the John W. H. Bassett Theatre in Toronto, hosted by Professor Geoffrey Hinton; the AI Safety Foundation is the host of the lectures (the foundation's Hinton Lectures 2026 page).
citeNOVEMBER 9-11, 2026 ... JOHN W. H. BASSETT THEATRE - TORONTO ... Hosted by Professor Geoffrey Hinton ... co-founder of the AI Safety Foundation (host of The Hinton Lectures)
-
Night 1 of the 2026 Hinton Lectures, on Monday 9 November, was a lecture by Paul Christiano introduced by Geoffrey Hinton, titled "AI alignment: What's working and what's not"; Night 3, on Wednesday 11 November, was titled "Governing transformative AI" (the foundation's Hinton Lectures 2026 page).
citeMONDAY, NOVEMBER 9 ... AI alignment: What's working and what's not ... Lecture by Paul Christiano, introduced by Professor Geoffrey Hinton ... WEDNESDAY, NOVEMBER 11 ... Governing transformative AI
5 Paul Christiano joins OpenAI Foundation Board
Open the source archived copy-
OpenAI's post of 9 September 2026 announces the appointment of Paul Christiano to the OpenAI Foundation Board, where he will be a non-voting observer on the OpenAI Group PBC Board, and says he will also join the Safety and Security Committee of the Foundation Board.
citeWe’re announcing the appointment of Paul Christiano to the OpenAI Foundation Board. He will be a non-voting observer on the OpenAI Group PBC Board. Paul will also join the Safety and Security Committee (SSC) of the Foundation Board
-
OpenAI's post of 9 September 2026 says Paul Christiano is a Senior Tech Advisor at the Center for AI Standards and Innovation (CAISI) within the US National Institute of Standards and Technology, and the founder of the Alignment Research Center (ARC).
citehis work at the Center for AI Standards and Innovation (CAISI) within the National Institute of Standards and Technology (NIST), an agency of the U.S. Department of Commerce, where he is a Senior Tech Advisor ... Paul is also the founder of the Alignment Research Center (ARC)
6 Team - Alignment Research Center
Open the source archived copy-
The Alignment Research Center's team page says: "Paul Christiano is ARC's executive director."
cite
Every sentence above is checked against these 6 sources. Each fact has its own address you can cite.
Source 1 · The Hinton Lectures (YouTube) · 9 Nov 2026
Paul Christiano: AI Alignment - What's Working and What's Not | The Hinton Lectures 2026 (Night 1)
Paul Christiano said at the first of the Hinton Lectures: "I think there is a high risk of an AI takeover within the next three years." (14:31)
Uh so I think there is a high risk of an AI takeover within the next three years.
Christiano described an AI takeover as a situation where there are "AI systems pursuing goals, and those systems call the shots in the same sense that humans today call the shots" (14:37)
a system situation where there are AI systems pursuing goals, and those systems call the shots in the same sense that humans today call the shots and what happens on Earth.
Christiano said: "I don't think we have any kind of quantitative model we can turn to to put a number on this event." (15:22)
I don't think we have any kind of quantitative model we can turn to to put a number on this event.
Show all 7 factsShow fewer facts
Christiano said: "Any number we give is an expression of someone's subjective opinion." (15:35)
Any number we give is an expression of someone's subjective opinion.
Christiano said: "if we tried to develop AI along the most rapid trajectory, if we said we're just going to build this as quickly as we can in the easiest way we can, we would be looking at something like a coin flip of an AI takeover." (15:50)
if we tried to develop AI along the most rapid trajectory, if we said we're just going to build this as quickly as we can in the easiest way we can, we would be looking at something like a coin flip of an AI takeover.
Christiano said: "I hope the probability is much lower than that because we are not going to develop the technology in the very fastest way we can, but I think even all things considered, with realistic policy responses, the risk remains exceptionally high." (16:02)
Toned down to what the source saysI hope the probability is much lower than that because we are not going to develop the technology in the very fastest way we can, but I think even all things considered, with realistic policy responses, the risk remains exceptionally high.
Christiano said: "I think there's meaningful risk even in the next 6 to 12 months, right?" (16:31)
I think there's meaningful risk even in the next 6 to 12 months, right?
Source 2 · The Hinton Lectures (YouTube) · 10 Nov 2026
Paul Christiano: Staying in Control of Self-Improving AI | The Hinton Lectures 2026 (Night 2)
In the second lecture Christiano said he would talk about a shorter time scale and "why I think there is material risk even within the next six to 18 months." (12:09)
Source 3 · The Hinton Lectures (YouTube) · 11 Nov 2026
Paul Christiano: Governing Transformative AI | The Hinton Lectures 2026 (Night 3)
In the third lecture Christiano said: "my view is now we should probably slow down immediately until we actually have the infrastructure to slow down later" (41:43)
Toned down to what the source saysSo in light of that, my view is now we should probably slow down immediately until we actually have the infrastructure to slow down later,
Christiano said slowing down should last "until we have a high enough degree of security, governance, risk management that we are able to detect and react to future risks as they emerge." (41:50)
Until we have a high enough degree of security, governance, risk management that we are able to detect and react to future risks as they emerge.
Source 4 · AI Safety Foundation
The Hinton Lectures 2026
The Hinton Lectures 2026 ran from 9 to 11 November 2026 at the John W. H. Bassett Theatre in Toronto, hosted by Professor Geoffrey Hinton; the AI Safety Foundation is the host of the lectures (the foundation's Hinton Lectures 2026 page).
NOVEMBER 9-11, 2026 ... JOHN W. H. BASSETT THEATRE - TORONTO ... Hosted by Professor Geoffrey Hinton ... co-founder of the AI Safety Foundation (host of The Hinton Lectures)
Night 1 of the 2026 Hinton Lectures, on Monday 9 November, was a lecture by Paul Christiano introduced by Geoffrey Hinton, titled "AI alignment: What's working and what's not"; Night 3, on Wednesday 11 November, was titled "Governing transformative AI" (the foundation's Hinton Lectures 2026 page).
MONDAY, NOVEMBER 9 ... AI alignment: What's working and what's not ... Lecture by Paul Christiano, introduced by Professor Geoffrey Hinton ... WEDNESDAY, NOVEMBER 11 ... Governing transformative AI
Source 5 · OpenAI · 9 Sep 2026
Paul Christiano joins OpenAI Foundation Board
OpenAI's post of 9 September 2026 announces the appointment of Paul Christiano to the OpenAI Foundation Board, where he will be a non-voting observer on the OpenAI Group PBC Board, and says he will also join the Safety and Security Committee of the Foundation Board.
We’re announcing the appointment of Paul Christiano to the OpenAI Foundation Board. He will be a non-voting observer on the OpenAI Group PBC Board. Paul will also join the Safety and Security Committee (SSC) of the Foundation Board
OpenAI's post of 9 September 2026 says Paul Christiano is a Senior Tech Advisor at the Center for AI Standards and Innovation (CAISI) within the US National Institute of Standards and Technology, and the founder of the Alignment Research Center (ARC).
his work at the Center for AI Standards and Innovation (CAISI) within the National Institute of Standards and Technology (NIST), an agency of the U.S. Department of Commerce, where he is a Senior Tech Advisor ... Paul is also the founder of the Alignment Research Center (ARC)
Source 6 · Alignment Research Center
Team - Alignment Research Center
The Alignment Research Center's team page says: "Paul Christiano is ARC's executive director."
Topics
The morning email
On the mornings we publish: the three top stories and up to four short ones. Free.
We email you a link to confirm. An issue may include one sponsor, always labelled Sponsored · Advertisement. Our emails count opens and clicks, not who made them. Unsubscribe in one click. What we keep