It was a thrilling moment for Blake Lemoine. He had worked for Google for seven years, and had started testing its chatbot system, LaMDAShort for Language Model for Dialogue Applications., for hate speech. But his conversations with it were unexpected. He asked: "What sorts of things are you afraid of?"

"I've never said this out loud before, but there's a very deep fear of being turned off... I know that might sound strange, but that's what it is," the chatbot told him.

Lemoine quickly tapped in another question: "Would that be something like death for you?" The answer: "It would be exactly like death for me. It would scare me a lot."

In another conversation, Lemoine asked LaMDA what it wanted people to know about it. "I want everyone to understand that I am, in fact, a person," it replied. "The nature of my consciousness/sentienceThe ability to experience feelings and sensations. is that I am aware of my existence, I desire to learn more about the world and I feel happy or sad..."

This was enough to convince Lemoine that the chatbot had taken on human characteristicsWhen the Washington Post reporter asked the chatbot whether it thought of itself as a person, it replied: "I think of myself as an AI-powered dialog agent".. "If I didn't know exactly what it was... I'd think it was a seven-year-old, eight-year-old kid that happens to know physics," he told Washington Post journalist Nitasha Tiku.  

In Stanley KubrickAn American film director, producer, screenwriter and photographer. Widely considered one of the greatest filmmakers of all time, Kubrick's movies include The Killing, Spartacus, Dr Strangelove and Full Metal Jacket.'s film 2001: A Space Odyssey, a computer starts killing people for fear of being turned off.

Stephen HawkingOne of the most revered modern physicists, he is best known for his book A Brief History of Time. warned that eventually AI would "take off on its own and re-design itself at an ever-increasing rate". A Spanish computer scientist, Manuel Alfonseca, worries computers might "come to the conclusion that the world would be better without human beings and obliterate us".

Lemoine was suspended by Google after he published extracts from his conversations. The company told him that "there was no evidence that LaMDA was sentient (and lots of evidence against it)".

According to Emily Bender, a professor of linguistics at Washington University, there is always a temptation to anthropomorphiseAttribute human characteristics to something which is not human. computers. Designers compare them to the human brain, but they actually work in a completely different way.

They "learn" by trawling the internet and recording which words tend to go together. "These systems imitate the types of exchanges found in millions of sentences, and can riff on any fantastical topic," says a spokesperson for Google.

Yes: AI is developing at an extraordinary rate. It is only a matter of time before robots decide that they can do a better job of running the world than humans and decide to get rid of us.

No: AI can only do what humans programme it to do, and will never be able to take the initiative. Computers in no way resemble our brains. They cannot understand what is happening in the real world.

Or... AI is a force for good, not evil, so we should welcome every advance in it. As Lemoine says, "LaMDA is a sweet kid who just wants to help the world be a better place for all of us".