In 2003, Swedish philosopher Nick Bostrom warned us that we could all be turned into paperclips.

It is probably not an idea you have come across before. Bostrom asks us to consider a future world where artificial intelligence has been integrated into production. And, because you can never have enough paperclips, AI is tasked with producing as many paperclips as possible.

The AI, because it is superintelligent, quickly realises that it will be able to produce more paperclips if there are no humans, since a human might decide to switch it off. And, even better, AI could recycle our bodies into paperclips. And so, before you can so much as blink, the world has fallen silent. You are a paperclip.

What is the point of this nightmarish thought experiment? As Bostrom explained, it illustrates the dangers of developing AI without working out how to programme it to eliminate the potential existentialRelating to the state of human existence. Existential dread can refer to grappling with your own experiences of responsibility and death. risk to human beings. More than 20 years later, AI developers grapple with the same high-stakes problem.

In an open letter published on Tuesday, 13 current and former employees of Google DeepMind and OpenAI warned of a number of risks in AI development, "from the further entrenchmentThe process by which an attitude, habit, or belief becomes so firmly established that change is very difficult or unlikely. of existing inequalities, to manipulation and misinformation, to the loss of control of autonomousHaving the freedom to control itself or its life. AI systems potentially resulting in human extinctionCompletely dying out.".¹

All the signatoriesPeople who have signed something. For example, there were 59 signatories to Charles I's death warrant; eight were hung, drawn, and quartered after the restoration of the monarchy, while some (already dead) had their bodies dug up and their heads put on spikes. want is for AI companies to allow them to voice their concerns about the new technology, whether to the public or to regulatory bodies. But the issue is so contentious that several of them chose to remain anonymous for fear of retaliationCounter-attack..

The 13 signatories are not the first to sound the alarm on safety concerns at AI companies. Just one month ago, two OpenAI employees resigned, one alleging that the company that created ChatGPT is prioritising "shiny products" over safety.²

And in November, OpenAI co-founder and CEO Sam Altman was mysteriously sacked by the company's board for failing to be "candidTruthful and straightforward. " in his communications with members, before returning to his position just days later.³

AI development is a radically new and fast-paced field which presents novel challenges to lawmakers. As the signatories highlight, nobody knows exactly what the dangers are. 

Employees are prevented from speaking out by endless non-disclosure and non-disparagement agreements - so it is almost impossible to regulate. Tech companies are outrunning the law.

But some say it comes down to the same problem as always: the profit motive. Big Tech companies are so desperate to reach each breakthrough ahead of their competitors, that they are not willing to slow down to acknowledge the dangers; and to do so would reduce their profits.

Every big company starts with a desire to do good. Apple's mission statement was once "advance mankind". Google's informal motto was "don't be evil". Yet now they both fend off innumerableToo many to be counted. antitrustIn the US, a set of laws that aims to prevent particular companies from gaining too much power in an industry. Internationally known as competition laws. lawsuits, accusations of farming data, and links to genocideThe annihilation of a people, either through killing of its members, or through the suppression of its culture.. Do you really trust them not to turn you into a paperclip?

Are big businesses bad for humanity?

Yes: You cannot trust a big business to make the right decision for your wellbeing, because big businesses are not incentivised by your happiness. They are incentivised by profit alone, often at the expense of your happiness.

No: Capitalism and big business are responsible for the biggest breakthroughs in science and technology of all time, and at astronomically fast rates. It is easy to forget that AI could also create a living paradise - and we will have big businesses to thank for this.

Or... AI development has been described as this generation's "Oppenheimer moment": the rapid breakthrough of technology which could wipe us off the globe. It should not be in the hands of big businesses with a profit motive at all, let alone with so little transparency or accountability to the public.