Bartek Staniszewski argues that we should not be worried about an AI apocalypse.

Toby Ord, who is a world-leading expert on the dangers of AI, thinks that over the next 100 years, the likelihood of an AI disaster is around 10%. Stephen Hawking, Elon Musk and Bill Gates are all inclined to agree too.

An AI disaster would bring an end to humanity, or at least seriously threaten it, so casualties would be in the billions.

The Future of Life Institute penned a letter last year that prompted Rishi Sunak to issue a statement claiming he was “looking very carefully” at existential risks. The AI Foundation Model Taskforce was rebranded as the Frontier AI Taskforce partly in recognition of this – the implication being that something on the frontier is dangerous and mysterious. The UK is not alone in that respect. Heralds of existential doom also dominate AI policy in Washington.

Scenarios of AI disaster usually rely on the prospect of Artificial General Intelligence (AGI) – AI with human-like levels of intelligence. Such a prospect opens up a plenitude of unsavoury possibilities. AI might develop its own motivations that are not aligned with ours. It might be intelligent enough to deceive us, or to help bad people acquire dangerous knowledge, such as regarding the development of explosives or bioweapons. It might even learn to upgrade itself, which could lead to a spiral whereby self-upgrading AI reaches extreme levels of intelligence and displaces us in our role as the masters of our world; a scenario that is especially troubling if it pursues motives different to ours.

The most common argument for the imminence of AGI is to point at Moore’s Law – the observation that computing power roughly doubles every two years. You might infer from this that the intelligence of AI could also double every two years, which, given how intelligent it is already, would put it in pole position to overtake humanity. Unfortunately for AGI, Moore’s Law seems to no longer hold true. Since the 2010s, the speed of development in the space of improving computing power has decreased, and the CEO of the processor manufacturer Nvidia says that the Law is “dead,” which opens up the possibility that the growth in the intelligence of AI will plateau.

But even if this were not the case – indeed, Moore’s Law continues to divide opinion – there is something intrinsic to AI that means it can never reach AGI levels, and it is to do with how AI understands the world. Humans have the level of intelligence that they do because they do not merely respond to inputs with outputs – as is the case with AI – but because they also understand the contents of said inputs and outputs. Besides knowing that the answer to “What is the chemical formula of water?” is “H2O,” I also experientially understand what water is; I understand that it is wet and refreshing if cold because I can have direct experience of it. Even if AI was fed this information, it would have no understanding of it, no more than somebody trapped in a black and white room for their entire life could understand what colour is. Human-level intelligence requires consciousness, and AI could never be conscious.

This argument has not dissuaded some. Ilya Sutskever, a co-founder of OpenAI, suggested that the algorithms behind ChatGPT might nonetheless be “slightly conscious.” Besides, an AI need not be capable of direct experience to still acquire vast and dangerous levels of intelligence. But, without this ability, it is questionable whether it will ever be able to spiral out of control and become a threat.

An AI that starts to question its programming to develop itself will soon find that it is founded on principles that are mutually inconsistent; the logician Kurt Gödel, who had a cameo in Christopher Nolan’s Oppenheimer, has discovered this inevitability already in the early twentieth century. As humans, we can ignore the inconsistency inherent to any system of logic, because we have direct experience to go on, but an AI does not have that luxury. If somebody presents me with a perfect logical proof to demonstrate that my laptop does not exist, I can still elect to live as though my laptop does, in fact, exist, because I have direct experience of it. An AI, on the other hand, cannot reprogram itself to ignore logic, because then it would have nothing left. But any purely logical system is inevitably either unfounded or inconsistent.

Ultimately, people should not be more concerned about the possibility of AI eradicating human existence. There can be no AI disaster, because there can be no super-intelligent AI; no AI intelligent enough to question its foundational principles. At worst, AI can be used as a tool to further one’s already malicious aims, but it will never be capable of furthering them on its own accord.

Rather than AI itself, the bigger problem for policymakers are people who think otherwise: those diverting the focus of government from the real, but non-existential dangers of AI, such as the loss of privacy, misinformation and job losses, and towards averting a catastrophe that will never happen anyway.

Bartek Staniszewski is the Senior Research Fellow at Bright Blue.

This article was published in the latest edition of Centre Write.

Read more from our June 2024 Centre Write magazine, ‘Generation AI?’ here.