How relying on computers leads to ‘disaster’

Trust issues: Google Home speaks a line from 2001 Space Odyssey. Apologies to Stanley Kubrick.

As Google launches its new smart home assistant, the economist Tim Harford warns that we are becoming too reliant on automation. When it fails, he says, the consequences are disastrous.

‘OK Google, dim the lights!’ ‘Turn up the music!’ ‘What’s a good substitute for caramel?’ ‘What’s the weather?’ ‘Do babies dream?’

When Google announced its new ‘Home’ device last week, these were just some of the commands and questions that it imagined people asking. The small white speaker is connected to ‘Google Assistant’, the company’s artificially intelligent personal helper, and it was launched to compete with the hugely popular Amazon Echo.

It works like this: when you say ‘OK Google’, the speakers light up and listen. Google Assistant has access to the company’s famous search engine, as well as your email account, calendar, Spotify, and ‘smart’ home features like thermostats. So when you say ‘tell me about my day’, it will give you a summary of the weather, some important news stories, and your appointments.

But a week after Google Home was announced, the author and journalist Tim Harford rang a 5,000 word alarm bell in The Guardian. We are becoming too reliant on automation, he warned — and it is ‘setting us up for disaster’. Just look at Air France Flight 447. When the plane’s autopilot failed and humans were forced to take control, they panicked. They were so used to a computer flying for them that their manual piloting skills were rusty. The plane was flying over a thunderstorm. Minutes later, it crashed into the Atlantic and killed everyone on board.

This is the ‘paradox of automation’, writes Harford. Most of the time computers are extremely useful, and so we gradually start to rely on them. But if they fail in extreme circumstances, we are left helpless at the very moment we need their help the most.

Of course, if your Google Home device fails it will not cost you your life. But as artificial intelligence becomes more and more entwined with daily life, the potential for mistakes only gets bigger.

Home truths

It’s worth the risk, say some. Humans are hardly immune from mistakes, and when it comes to certain things — like mental maths, keeping track of birthdays, and maybe even driving cars — computers are far more reliable. Allowing them to do these routine jobs gives us more time to focus on the things that humans do better: being creative, solving real-world problems, and talking to other humans.

That is fine 99.9% of the time, says Harford. But if we put too much trust in computers, things are far worse when they do go wrong, as we are totally unable to rely on our own wits. The only solution is to flip our relationship with computers on its head. Rather than letting them do our jobs for us, we should do them ourselves and use computers to make sure we haven’t missed anything, or to step in when things get hairy.

You Decide

  1. Would you like a Google Home for your bedroom?
  2. Are we becoming too dependent on computers to do things for us?


  1. Rewrite the final two paragraphs of this story, presenting two sides of a different argument about Google Home. If you’re unsure, think about privacy concerns, artificial intelligence, or whether Google has become too powerful.
  2. Challenge yourself by not using computers for an entire day — that includes mobile phones, game consoles, and tablets. Record your experiences in a diary.

Some People Say...

“Never trust a computer with something you cannot do yourself.”

What do you think?

Q & A

I’m not a pilot. I don’t have Google Home. Why should I worry about this?
Flying a plane is an extreme example, but there are plenty of common skills which have been lost through computers. When was the last time you memorised a phone number? Or worked out the best route to a destination? These are relatively small things. But if Google Home and Amazon Echo catch on, they will represent a shift in how much we use technology — and they will only get more powerful.
They look great! How can I get one?
Amazon Echo was launched in the UK last month priced at £149.99. Google Home will be available in America from November 4th, costing $129. It is not clear yet when it will reach Britain or how much the UK price will be, but it will be cheaper than the Echo (this is one benefit of coming second).

Word Watch

Google Assistant
Similar to Apple’s ‘Siri’ or Amazon’s ‘Alexa’, this is an AI bot which will soon be found across many different Google devices. The company has been working with comedians from Pixar and the satirical website The Onion to make its voice funny and easy to talk to.
Amazon Echo
These black speakers were released last year, and they can also perform similar actions, such as giving local news alerts, ordering things from Amazon, or giving you ‘interesting facts’. By April this year, around three million had been sold.
Air France Flight 447
The flight, from Rio to Paris, took place on June 1st 2009.
Computers can automatically fly planes from point A to B with very little human intervention. In the case of Flight 447, however, one of the sensors iced over and disconnected at a crucial moment. Another automated system, which helps to smooth out manual piloting controls, also failed at the same time.
Driving cars
Self-driving cars are being developed by tech firms Google, Apple and Uber, as well as more traditional car companies.


PDF Download

Please click on "Print view" at the top of the page to see a print friendly version of the article.