They want to ban Artificial Superintelligence. But we have no idea what that means.

blog@dws.team
October 22, 2025
about 2 months ago
They want to ban Artificial Superintelligence. But we have no idea what that means.

A facile commitment because the danger of intelligent machines is subject to deflation.

A facile commitment because the danger of intelligent machines is subject to deflation.

The proposed ban is like the proverbial man with the red flag. Mandated by law to walk ahead of the first automobiles, he was soon set aside as the nature of speed was better understood. Once we get closer to ASI, our definitions will have developed and changed.

Vehicles in the times of the red flag were hulking big nothing-burgers. By the time they got useful, opposition had moved on to other subjects.

The Guardian this week made big news out of Prince Harry and Meghan setting themselves behind a ban on Artificial Superintelligence. Together with a bunch of Nobel Prize laureates and others anxious to remain relevant.

We differentiate between Artificial Intelligence, Artificial General Intelligence, and Artificial Superintelligence. Where the last on the list is a machine that can improve itself. Think The Matrix.

The current fascination is with a form of Artificial Intelligence that creates strings of words using neural networks powered by statistics and big data. It reproduces the components of language in the order in which it is most likely to be found in the context of a given prompt. We assign it the qualification “intelligence” much like we qualify the answer it gives as helpful. The critical part being that it's us doing the qualifying.

AI is highly useful. All sorts of tasks can be done much more quickly than before, tasks that we now discover are repetitive and boilerplate and actually, on second thought, not much worth the attention of a human. Summarising, translating, even writing code: it deflates earlier human involvement like currency inflation so out of control that new bills must be printed every week.

The concept of intelligence itself is subject to a similar deflation. And that's not a first. Compulsory education delivered humans to work in great halls resounding with the clank of typewriters, only to be replaced by office printers, to be replaced in turn by email and websites. Reproduction of language was then deemed too simple for the attention of a human: they should instead be creative. Oh but wait, can’t we replace that as well? Right.

Turns out then, that “being creative” needs to be redefined. Another term subject to deflation.

Sam Altman is toning down the timeline of OpenAI regarding Artificial General Intelligence. It’s no longer “very soon”, “this decade”. Smart move. He is uniquely positioned to understand what AI turns out not to be.

As we understand better what the actual real-world tasks entail that our current AI can replace, we are beginning to understand how fragmentary our understanding is of what it means to be intelligent.

So, to the signees I say this: it’s the least impactful commitment you've ever given. By the time we get closer to the machine you fear so much, the term “superintelligence” will have devolved like the German Mark during the hyperinflation of the 20's: not worth the paper it's printed on.