Aiodic

Back to the table

117 Emerging Si Superintelligence Frontier

Superintelligence

AI far beyond human ability.

What it is

Superintelligence is the idea of an AI system that is far more capable than the best human beings at nearly everything that involves thinking: science, strategy, persuasion, engineering, planning, invention. Not better at one game or one task, but better across the board, and by a wide margin.

The most quoted definition comes from the Oxford philosopher Nick Bostrom, who in his 2014 book Superintelligence described it as "any intellect that greatly exceeds the cognitive performance of humans in virtually all domains of interest". Notice what that definition leaves out. It says nothing about whether the machine would be conscious, have feelings, or look like anything in a film. It is only about capability: what the system could do.

The first thing to say plainly is that no superintelligent system exists. Everything on this page is about a possible future, and people who know the field well disagree sharply about whether, when, and how it could arrive. That disagreement is the heart of the topic, so this page tries to show it fairly rather than settle it.

Narrow superhuman ability is already here

It helps to separate two things that often get blurred together.

Narrow superhuman ability means a machine that beats every human at one specific job. This has existed for decades. In May 1997 IBM's Deep Blue beat the world chess champion Garry Kasparov in a six-game match in New York, 3.5 to 2.5. In March 2016 DeepMind's AlphaGo beat Lee Sedol at Go, and its famous move 37, a move experts first took for a mistake, is described on the Reinforcement Learning page. In 2020 DeepMind's AlphaFold predicted the shapes of proteins with accuracy that had eluded scientists for decades, work that shared the 2024 Nobel Prize in Chemistry (the story is on the Artificial Intelligence page).

Each of these systems is superhuman inside a fenced-off field and helpless outside it. AlphaGo could not play chess, write an email, or notice that the building was on fire.

General superintelligence would be different in kind: one system (or a team of systems) that outperforms people at almost everything at once, including the job of doing AI research. It sits one step beyond AGI, which usually means AI that matches a capable human across most tasks. You can think of the ladder as narrow AI, then human-level general AI, then something well beyond human level. Today's chatbots are somewhere interesting on the first rungs: remarkably broad, yet still prone to hallucinations and to failing at things a child finds easy.

The "intelligence explosion" idea

The reason superintelligence gets discussed at all, rather than treated as science fiction, goes back to a 1965 paper by the British mathematician I. J. Good, who had worked with Alan Turing at Bletchley Park. In "Speculations Concerning the First Ultraintelligent Machine", Good reasoned that designing machines is itself an intellectual task. So a machine better than any human at intellectual tasks could design a better machine, which could design a better one still. He called the result an "intelligence explosion", and wrote that the first such machine would be "the last invention that man need ever make, provided that the machine is docile enough to tell us how to keep it under control."

That one sentence contains both halves of the modern debate: the hope (a machine that solves our problems) and the worry (the condition about keeping it under control).

Why some people think it could be near

People who expect superintelligence relatively soon usually point to three things. Capabilities have grown quickly as companies have used more computing power and data. AI systems are increasingly used to help write code and run experiments, including AI research itself, which is the loop Good described. And several heads of AI companies have said publicly that they expect very powerful systems within years rather than decades, as the examples below show. It is worth remembering that these are forecasts, and some come from people whose companies raise money on the strength of them.

Why others doubt it

Sceptics make several different arguments, and they're worth keeping apart. Some, like the Meta and later AMI Labs researcher Yann LeCun, argue that today's methods are missing basic abilities, such as understanding the physical world and planning, so talk of superintelligence is premature (see World Models). Others argue that "intelligence" isn't a single dial that can be turned up without limit. And others accept that AI will keep improving but argue that its effects will arrive slowly, because real organisations, laws, and markets take years or decades to change. None of these views says AI is harmless. They say the superintelligence story is the wrong way to think about the risks.

The control problem in plain words

The worry, in its simplest form, is this. If you build something much more capable than you, you need it to want what you want, and you need a way to correct it if it doesn't. Both are hard. It is already difficult to make today's systems follow intentions rather than literal instructions, which is the subject of Alignment. With a system that can out-think its overseers, mistakes might be harder to spot and harder to undo. That is the "control problem". Whether it would actually be unsolvable, or would be handled step by step like other engineering risks, is exactly what the two camps disagree about.

Why it matters

It shapes where enormous sums of money go. Superintelligence is no longer only a philosopher's word. Companies now put it in their names and mission statements, and investors fund them on that basis. When a company says it is building superintelligence, it helps to know what the word claims and how uncertain it is.

It drives safety research and law. Much of the work on alignment, interpretability, and testing, and several recent laws, exist partly because of worries about very capable future systems. You don't have to share those worries to see that they shape the rules your AI tools are built under.

It competes for attention with today's harms. Some critics argue that talk of far-future superintelligence distracts from present problems such as bias, privacy, misinformation, and concentration of power. Others reply that you can work on both. Knowing the argument helps you judge which voices you're hearing.

It's where headlines get most emotional. Stories about superintelligence swing between salvation and doom. A calm reader who can tell a forecast from a fact is harder to frighten and harder to sell to.

Serious people genuinely disagree. Among the researchers most associated with modern AI, Geoffrey Hinton and Yoshua Bengio have warned about the risks, while Yann LeCun, who shared the 2018 Turing Award with them, has called existential fears overblown. When experts at that level disagree, the honest position for the rest of us is to hold views loosely and watch the evidence.

Real-world examples

From a 1965 thought experiment to a 2014 bestseller

I. J. Good's paper was published in 1965 in the series Advances in Computers. For decades the idea stayed mostly in academic papers and science fiction. That changed with Nick Bostrom's Superintelligence: Paths, Dangers, Strategies, published by Oxford University Press in July 2014. Bostrom set out the possible routes to superintelligence, argued that a system far more capable than us could be very hard to control once built, and made the case that the control problem should be worked on before it was needed. The book became a bestseller and moved the idea from the edges of philosophy into conversations among technology leaders. It also drew strong criticism from researchers who thought it was built on speculation about systems nobody knew how to make. Both reactions still shape the debate today.

Two statements in one week (May 2023)

Six months after ChatGPT launched, the debate went mainstream. On 22 May 2023, OpenAI's Sam Altman, Greg Brockman, and Ilya Sutskever published a post titled "Governance of superintelligence". They wrote that it was conceivable that within the next ten years AI systems would exceed expert skill level in most domains, and suggested that the world might eventually need an international body for the most powerful systems, along the lines of the International Atomic Energy Agency.

On 30 May 2023, the Center for AI Safety, a nonprofit in San Francisco, published a one-sentence statement: "Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war." Its signatories included Geoffrey Hinton, Yoshua Bengio, and the heads of OpenAI (Sam Altman), Google DeepMind (Demis Hassabis), and Anthropic (Dario Amodei), alongside hundreds of academics and others. Hinton had announced at the start of May 2023 that he had left Google, saying he wanted to be able to talk about the dangers of AI freely.

It's worth reading the statement carefully. It doesn't say extinction is likely, or when. It says the risk deserves priority. Critics at the time pointed out that some signatories were building the very technology they warned about, and asked why they didn't simply slow down.

"Superintelligence" becomes a business plan (2024 to 2025)

On 19 June 2024, Ilya Sutskever, OpenAI's cofounder and former chief scientist, announced a new company, Safe Superintelligence Inc. (SSI), with Daniel Gross and Daniel Levy. Its website describes building safe superintelligence as "the most important technical problem of our time" and says the company has one goal and one product, so that safety is insulated from short-term commercial pressure.

A year later, on 30 June 2025, Mark Zuckerberg sent Meta staff a memo creating Meta Superintelligence Labs, led by Alexandr Wang, the former chief executive of Scale AI, with Nat Friedman. In the memo Zuckerberg wrote that developing superintelligence was "coming into sight". Daniel Gross left SSI for Meta around the same time.

On 10 June 2025, Sam Altman opened a blog post called "The Gentle Singularity" with the words "We are past the event horizon; the takeoff has started", and argued that humanity was close to building digital superintelligence. In the same post he listed solving the alignment problem as the first priority. Dario Amodei of Anthropic avoids the word, but in his October 2024 essay "Machines of Loving Grace" he wrote that very powerful AI, which he described as a "country of geniuses in a datacenter", could arrive "as early as 2026, though there are also ways it could take much longer."

These are statements of belief and intent from people who lead companies competing for talent and investment. They tell you what these leaders are aiming for and say they expect. They are not evidence that superintelligence is coming on any particular date.

The Statement on Superintelligence (October 2025)

On 22 October 2025, the Future of Life Institute, a nonprofit that campaigns on risks from powerful technologies, published a one-sentence call for "a prohibition on the development of superintelligence, not lifted before there is broad scientific consensus that it will be done safely and controllably, and strong public buy-in."

The first signatories were an unusually mixed group. They included the AI researchers Geoffrey Hinton, Yoshua Bengio, and Stuart Russell; Apple cofounder Steve Wozniak; Virgin founder Richard Branson; former Irish president Mary Robinson; former US national security adviser Susan Rice; Prince Harry and Meghan, the Duke and Duchess of Sussex; and the conservative commentator Steve Bannon. The institute released it with a poll it had commissioned of 2,000 US adults, in which 64 percent agreed superintelligence shouldn't be developed until it was provably safe and controllable, or should never be developed.

Critics raised practical objections: nobody has an agreed test for when a system counts as superintelligent, so a ban would be hard to define or enforce, and it could hold back useful research. Supporters replied that the difficulty of drawing a line is a reason to start the conversation, not to skip it. At the time of writing the statement's website listed tens of thousands of signatures; check the site for the current figure.

The sceptics make their case (2024 to 2025)

In October 2024, Yann LeCun, then Meta's chief AI scientist, was asked by the Wall Street Journal whether AI could become smart enough to threaten humanity. He replied, in words the paper printed, that it was "complete B.S." His argument, made many times, is that current systems still lack abilities that even a house cat has, such as understanding the physical world, planning, and lasting memory, so worrying about controlling a superintelligence is getting far ahead of the science.

On 15 April 2025, the Princeton computer scientists Arvind Narayanan and Sayash Kapoor published "AI as Normal Technology" through Columbia University's Knight First Amendment Institute. They argue that AI is best understood like electricity or the internet: powerful, but spreading through society over decades, because businesses, regulators, and people adapt slowly. They are sceptical that "superintelligence" is even a coherent idea, and argue that what matters is power over the world, which AI would have to be given, step by step, by people who can watch how it behaves. They don't say there is nothing to worry about. They argue that the more probable risks are things like inequality, surveillance, and concentrations of power, and that the best defence is resilience: many layers of safeguards, monitoring, and the ability to reverse decisions.

What governments and labs say they're doing (2023 to 2026)

Governments have mostly avoided the word "superintelligence" and talk instead about "frontier" or "general-purpose" AI. After the first AI Safety Summit at Bletchley Park in November 2023, 30 countries backed an International AI Safety Report. The first edition, written by 96 experts led by Yoshua Bengio, was published on 29 January 2025. The 2026 edition, published in February 2026, says it is plausible that AI progress up to 2030 could slow, continue at the current pace, or speed up dramatically, and treats loss of control as a scenario experts take seriously while noting that today's systems don't have the abilities to cause it.

Laws have followed, focused on transparency and testing rather than bans. In the European Union, obligations for providers of the most capable general-purpose AI models, including assessing and reducing "systemic risk", started to apply on 2 August 2025. In California, Governor Gavin Newsom signed SB 53, the Transparency in Frontier Artificial Intelligence Act, on 29 September 2025; it requires the largest developers to publish how they manage catastrophic risks and to report serious safety incidents. The UK renamed its AI Safety Institute the AI Security Institute in February 2025, focusing its testing on security risks such as cyber-attacks and weapons.

The big AI companies, for their part, publish safety frameworks that say which tests a model must pass before release and what precautions increase as models become more capable, and they run alignment teams. How well these promises hold up under commercial pressure is itself debated; the Alignment page tells the story of OpenAI's short-lived Superalignment team. Rules and frameworks change quickly, so check the current versions if they matter to you.

Try it yourself

You don't need to decide whether superintelligence is coming to think clearly about it. This exercise trains the one skill that helps most: sorting claims into evidence, prediction, and opinion, and noticing who is speaking. It takes about 30 minutes and works with any AI chatbot or none at all.

  1. Collect three claims. Search a news site for "superintelligence" and pick three recent articles. Copy one key sentence from each into a document. Try to get a mix: one from an AI company leader, one from a researcher or campaigner, one from a sceptic.
  2. Sort each claim. For each sentence, write one label:
  1. Ask who is speaking and what they gain. Next to each claim, note the speaker's job and one way the claim might benefit them, whether money, attention, funding, influence over rules, or simply being proved right. This isn't about assuming bad faith. Everyone has a position, including critics.
  2. Get a second opinion on your sorting. Paste this into a chatbot, adding your three sentences: “` Here are three statements about superintelligence from news articles. For each one, split it into parts that are (a) checkable evidence, (b) predictions about the future, and (c) opinions or value judgements. Don't tell me whether the claim is right. Just sort it, and say what evidence would change a reasonable person's mind about each prediction.
  3. [paste statement]
  4. [paste statement]
  5. [paste statement] “`
  6. Ask for the strongest case on each side. Then try: “ Give me the strongest good-faith argument that superintelligence is a serious near-term concern, and the strongest good-faith argument that it is not. Name real people or organisations who hold each view, with dates, and tell me which parts are facts and which are forecasts. “ Chatbots can invent quotes and dates, so check any name or date it gives you by searching for the original source before you rely on it (see Hallucination).
  7. Write down your own position, with a confidence level. One sentence, such as "I think it's unlikely in the next ten years, about 70 percent sure." Add what would change your mind. Come back to it in six months and see whether anything did.

The point isn't to reach the right answer. It's to notice that most of what you read about superintelligence is prediction and opinion, and to treat it accordingly.

Common mistakes and misunderstandings

Thinking it already exists. Today's chatbots are impressive and broad, but they are not superintelligent. They still make basic errors and need human checking. Superhuman performance in chess, Go, or protein folding is narrow, not general.

Treating a prediction as a fact. "Superintelligence is coming by 2030" and "superintelligence is impossible" are both forecasts. Nobody has data from the future. Surveys show how divided experts are: in a survey of 2,778 AI researchers published in January 2024, the combined forecast gave a 50 percent chance that machines could outperform humans at every task by 2047, with individual answers spread across decades, and many respondents put a small but real probability on extremely bad outcomes. A spread that wide is itself the main finding.

Assuming superintelligence means a robot with feelings. The debate is about capability, not consciousness. A system could be extremely capable without having any inner life, and a worried researcher isn't claiming machines will hate us.

Dismissing all concern as hype, or all scepticism as naivety. Some warnings come from people with commercial interests, and some scepticism does too. Some of the most worried and some of the most sceptical voices are among the most respected researchers in the field. Judge the argument, not the tribe.

Thinking it's all or nothing. You'll often see the choice framed as "utopia or extinction". Most serious analyses, including the International AI Safety Report, consider a range of outcomes, and many risks from powerful AI, such as misuse, accidents, and concentrated power, don't depend on superintelligence at all.

Believing nothing can be done. Governments and companies are testing, reporting, and regulating, imperfectly. Researchers work on alignment, interpretability, and guardrails. Whether that is enough is debated, but "nobody is doing anything" isn't accurate.

Related elements