How relying on computers leads to ‘disaster’

As Google launches its new smart home assistant, the economist Tim Harford warns that we are becoming too reliant on automation. When it fails, he says, the consequences are disastrous. ‘OK Google, dim the lights!’ ‘Turn up the music!’ ‘What’s a good substitute for caramel?’ ‘What’s the weather?’ ‘Do babies dream?’ When Google announced its new ‘Home’ device last week, these were just some of the commands and questions that it imagined people asking. The small white speaker is connected to ‘Google Assistant’, the company’s artificially intelligent personal helper, and it was launched to compete with the hugely popular Amazon Echo. It works like this: when you say ‘OK Google’, the speakers light up and listen. Google Assistant has access to the company’s famous search engine, as well as your email account, calendar, Spotify, and ‘smart’ home features like thermostats. So when you say ‘tell me about my day’, it will give you a summary of the weather, some important news stories, and your appointments. But a week after Google Home was announced, the author and journalist Tim Harford rang a 5,000 word alarm bell in The Guardian. We are becoming too reliant on automation, he warned — and it is ‘setting us up for disaster’. Just look at Air France Flight 447. When the plane’s autopilot failed and humans were forced to take control, they panicked. They were so used to a computer flying for them that their manual piloting skills were rusty. The plane was flying over a thunderstorm. Minutes later, it crashed into the Atlantic and killed everyone on board. This is the ‘paradox of automation’, writes Harford. Most of the time computers are extremely useful, and so we gradually start to rely on them. But if they fail in extreme circumstances, we are left helpless at the very moment we need their help the most. Of course, if your Google Home device fails it will not cost you your life. But as artificial intelligence becomes more and more entwined with daily life, the potential for mistakes only gets bigger. Home truths It’s worth the risk, say some. Humans are hardly immune from mistakes, and when it comes to certain things — like mental maths, keeping track of birthdays, and maybe even driving cars — computers are far more reliable. Allowing them to do these routine jobs gives us more time to focus on the things that humans do better: being creative, solving real-world problems, and talking to other humans. That is fine 99.9% of the time, says Harford. But if we put too much trust in computers, things are far worse when they do go wrong, as we are totally unable to rely on our own wits. The only solution is to flip our relationship with computers on its head. Rather than letting them do our jobs for us, we should do them ourselves and use computers to make sure we haven’t missed anything, or to step in when things get hairy. KeywordsAmazon Echo - A series of speakers produced by Amazon that are voice-activated and can be asked to carry out tasks. They are powered by Alexa, a personal assistant AI developed by Amazon.