In 2003, Swedish philosopher Nick Bostrom warned us that we could all be turned into paperclips.

It is probably not an idea you have come across before. Bostrom asks us to consider a future world where artificial intelligence has been integrated into production. And, because you can never have enough paperclips, the AI is tasked with producing as many paperclips as possible.

AI, because it is superintelligent, quickly realises that it will be able to produce more paperclips if there are no humans, since a human might decide to switch it off. And, even better, it can recycle our bodies into paperclips. And so, before you can so much as blink, you are a paperclip.

What is the point of this nightmarish thought experiment? As Bostrom explained, it illustrates the dangers of developing AI without working out how to programme it to eliminate the potential existentialRelating to the state of human existence. Existential dread can refer to grappling with your own experiences of responsibility and death. risk to human beings. More than 20 years later, AI developers grapple with the same problem.

In an open letter published on Tuesday, 13 current and former employees of Google DeepMind and OpenAI warned of a number of risks in AI development, "from the further entrenchmentThe process by which an attitude, habit, or belief becomes so firmly established that change is very difficult or unlikely. of existing inequalities, to manipulation and misinformation, to the loss of control of autonomousHaving the freedom to control itself or its life. AI systems potentially resulting in human extinctionCompletely dying out.".¹

All the signatoriesPeople who have signed something. For example, there were 59 signatories to Charles I's death warrant; eight were hung, drawn, and quartered after the restoration of the monarchy, while some (already dead) had their bodies dug up and their heads put on spikes. want is for AI companies to allow them to voice their concerns about the new technology, whether to the public or to regulatory bodies. But the issue is so contentious that several of them remained anonymous for fear of retaliationCounter-attack..

AI development is a radically new and fast-paced field which presents novel challenges to lawmakers. As the signatories highlight, nobody knows exactly what the dangers are. 

Employees are prevented from speaking out, so it is almost impossible to regulate. Tech companies are outrunning the law.

But some say it comes down to the same problem as always: the profit motive. Big Tech companies are so desperate to reach each breakthrough ahead of their competitors, that they are not willing to slow down to acknowledge the dangers.

Every big company starts with a desire to do good. Apple's mission statement was once "advance mankind". Google's informal motto was "don't be evil". Yet now they both fend off innumerableToo many to be counted. antitrustIn the US, a set of laws that aims to prevent particular companies from gaining too much power in an industry. Internationally known as competition laws. lawsuits, accusations of farming data, and links to genocideThe annihilation of a people, either through killing of its members, or through the suppression of its culture.. Do you really trust them not to turn you into a paperclip?

Are big businesses bad for humanity?

Yes: You cannot trust a big business to make the right decision for your wellbeing, because big businesses are incentivised by profit alone, often at the expense of human happiness.

No: Capitalism and big business are responsible for the biggest breakthroughs in science and technology of all time, and at astronomically fast rates. It is easy to forget that AI could also create a living paradise - and we will have big businesses to thank for this.

Or... AI development has been described as this generation's "Oppenheimer moment": the rapid breakthrough of technology which could wipe us off the globe. It should not be in the hands of big businesses with a profit motive at all, let alone with so little transparency or accountability to the public.

Existential - Relating to the state of human existence. Existential dread can refer to grappling with your own experiences of responsibility and death.

Entrenchment - The process by which an attitude, habit, or belief becomes so firmly established that change is very difficult or unlikely.

Autonomous - Having the freedom to control itself or its life.

Extinction - Completely dying out.

Signatories - People who have signed something. For example, there were 59 signatories to Charles I's death warrant; eight were hung, drawn, and quartered after the restoration of the monarchy, while some (already dead) had their bodies dug up and their heads put on spikes.

Retaliation - Counter-attack.

Innumerable - Too many to be counted.

Antitrust - In the US, a set of laws that aims to prevent particular companies from gaining too much power in an industry. Internationally known as competition laws.

Genocide - The annihilation of a people, either through killing of its members, or through the suppression of its culture.