Predicting the future is a risky business. If it wasn’t, we’d all be very wealthy by now. The Danish physicist Neils Bohr famously opined: “Prediction is very difficult, especially about the future”.
Despite this, I confidently predict that machines will come to run our lives. And I’m not alone in this view. US mathematician Claude Shannon, one of the fathers of computation, wrote: “I visualize a time when we will be to robots what dogs are to humans, and I’m rooting for the machines.”
And physicist Stephen Hawking, who is never short of a quote on life, the universe and everything has said that: “Unless mankind redesigns itself by changing our DNA through altering our genetic makeup, computer-generated robots will take over our world”.
So how can we be so sure? Well, in a sense, it’s already happened. Computers are in charge of many aspects of our lives and it’s probably too late to turn them off.
Last month, medical bills in Australia couldn’t be paid. The cause? Computer software in the Australian Health Industry Claims and Payments Service (HICAPS) system that didn’t know about the leap day.
In November 2009, the entire air traffic control system of the United States crashed, causing chaos to travellers. The cause? The failure of a single router board.
And in August 2003, a powercut in the United States put 55 million people in the dark. The cause? Faulty software on a single computer that failed to detect what should have been a harmless local outage.
And there are many more examples. When computers fail, we see just how dependent we have become on them.
Historians will probably look back from the 22nd century and observe that the rise of machines became inevitable the day we first picked up a rock and started using it as a tool. Since then, we’ve been using machines to amplify our physical and, more recently, our mental capabilities.
Computers are now embedded into almost every aspect of our lives. Sometimes they’re even making life and death decisions:
In the 1980s, a bug in the software of the Therac-25 radiation therapy machine was directly responsible for at least five deaths.
In 1991 in Saudi Arabia, a bug in the software of the MIM-104 Patriot surface-to-air missile system led to an incoming Scud missile not being located, and indirectly to the death of 28 people.
In 2007 in South Africa, a robotic anti-aircraft cannon accidentally killed nine people and injured 14 others.
Given these incidents (and others), it is unsurprising there is concern in some quarters about the risk of giving up control to machines. As a scientist, I welcome this discussion.
Roman Yampolskiy, a computer scientist at the University of Louisville in Kentucky, recently joined this debate with an article in the March issue of the Journal of Consciousness Studies (yes, such a scholarly tome does exist).
Yampolskiy proposed that any artificial intelligence we develop should be confined within a secure computing environment. In practical terms, this could mean severely limiting the ability of the AI to interact with the outside world. The AI would live in a virtual “prison”.
Confining AI in this way would prevent harmful effects since the computer would not be able to take direct actions, only offer advice. However, it would still allowing humanity to benefit from the AI’s super-intelligence.
This might sound like a good idea, but there are many arguments against this strategy.
First, it’s probably not possible. Where mankind has faced other, similar threats, confinement has been a controversial option.
For instance, while the smallpox virus is now confined to just two laboratories around the world, many believe this leaves us exposed to bioterrorist threats.
And cinema is full of examples where artificial intelligence manages to escape any such controls – think of films such as Blade Runner, The Matrix series and The Terminator series. Sure, these are just films, but fiction has a terrible habit of becoming fact. Our imaginations are often the best tool we have for predicting the future.
Second, confining AI is not desirable. Artificial intelligence can help us tackle many of the environmental, financial and other problems facing society today. This just won’t be possible if we isolate machines. If you isolate a child, they will struggle to learn and develop intelligence.
Many scientists, myself included, believe intelligence doesn’t exist in isolation, but emerges from our interaction with the ever-changing world.
Third, confining AI creates a false sense of security. Isaac Asimov had the right idea here: we need to ensure the DNA of any machine is designed to prevent harm. Asimov’s First Law of Robotics – which appeared in his 1942 short story, Runaround – states:
“A robot may not injure a human being or, through inaction, allow a human being to come to harm.”
Like all technologies, computers offer immense potential for good and for bad. It is our duty to properly train the next generation of computer scientists so “good” is programmed into the very DNA of future computers.
The Lazy Koala
logged in via Twitter
Interesting article, although I think you might be overstating the risks computers pose, especially contemporary ones.
For instance, "a powercut in the United States put 55 million people in the dark. The cause? Faulty software" - this is a very strange example. Technology failed and as a result we lost access to our technology. This is equivalent to blaming spears for an incident in which someone lost their spear and then was attacked by a bear.
Similarly with not being able to pay health bills or get on a plane. In other words, when technology fails, we lose access to that technology. It doesn't make us servants or useless. We can still eat, live and go to the beach.
These computer failures might seem like a big deal these days, but humans managed fine for hundreds of thousands of years without these minor conveniences.
If the robots fight back, I think we can nuke them and go to the beach. No worries.
Toby Walsh
Professor, Research Director at NICTA
It would be nice to just go down the beach, but with the ever increasing population of our planet, this isn't just possible any more.
We depend on technology, computers especially, to provide the efficiencies that support our population. So sadly, we couldn't still eat and live without technology.
James Walker
logged in via Facebook
We are either dependent on the computerised system, or not.
If it is possible to solve a problem without using the computers, then we are capable of having a back up system for when they fail us. This was normal when computers were first introduced; paper trails etc were still required.
If it is *not* possible to solve a problem without computers, then it is pointless to fret about the system failing: the benefits cannot be acquired any other way.
Either way: "I for one welcome our new robot overlords."
Toby Walsh
Professor, Research Director at NICTA
Even if the benefits can be acquired no other way than with computers, we can still fret usefully about how we design those computers. Some designs will be more robust and reliable than others.
Emma Anderson
Artist and Science Junkie
Funny you should say that because in a mechanistic view of the universe, it might be said that we're actually robots.
Chris Harries
logged in via Facebook
A month ago I dropped into a large printery in town to order a job, artwork in hand. "Sorry, we can't take your order, the computer is down" said the person at the desk. I was somewhat nonplussed, thinking, surely a pen and a writing pad could take down my order for the time being, I was in no rush for the job to be done. But no, she helpfully directed me to a competitor three blocks away.
The problem there, as I see it, is that our dependence on computer systems leaves us incapable of functioning…
Read moreToby Walsh
Professor, Research Director at NICTA
I agree. We often become dependent on our (technological) crutches.This is another risk of computers encroaching more and more into our lives. I sometimes observe at the checkout counter waiting for my change that even a simple skill like subtraction is dying out!
William Ferguson
Software Developer
It's not that we are incapable of acting without them, it's just that the business models that they have allowed us to use become uneconomic without them.
In the example cited, the low cost margins on bespoke printing today mean that taking a manual order become non-profitable leading to the suggestion of using the competitor.
Likewise while it is possible to manually count sheep, share trades, stars in the sky, the expense associated makes such a manual task uneconomic in most scenarios.
IMHO it's not really an us or them decision. Many of the tools that will be available to AI will also be available to humanity and we will find ourselves augmenting our abilities over time. Just like we currently augment our abilities by wearing clothes, using machines that lift and move mega-tonnages and communicate and calculate at whim.
I for one will celebrate the day when the majority of the cars on our roads are driven by AI with the associated reduction in road carnage.
Chris Harries
logged in via Facebook
No William, the person at the desk was employed there regardless, with nothing to do. In that case there was no economic imperative not to use her time to take down my order.
Re your last line, here's a bold assumption that technical measures would necessarily reduce road carnage. The Fukushima meltdown was not meant to happen. In fact we were told that such an event was virtually impossible. Why did the world have to invest 31 trillion dollars in remedying the Y2K mistake?
Adding layers of…
Read moreWilliam Ferguson
Software Developer
Chris, you're making a very facile assumption about the business processes in use at the printers. For the order to be taken manually requires at the least that she know all the right questions to ask for your particular type of order (which could easily run into the thousands of variations). If the computers don't come back online she may then need to create several documents, send some of them thousands of kilometres and may not even be able to start without your signature on all the documentation…
Read moreChris Harries
logged in via Facebook
Well William, I think that supports exactly what the author has written. If a simple computer breakdown causes such severe implications then mankind has lost control over its destiny. Once upon a time technology was there to serve us. But there is a threshold beyond which it controls our lives more than it serves us. Whether we have got to that stage yet is a moot point. The good professor is challenging us to be aware of that threshold. I do believe that the risks are largely invisible, that is until we find that something has gone terribly wrong.
William Ferguson
Software Developer
Actually my read of the article is that Professor Walsh is urging us to investigate and build good mechanisms into our software tools to mitigate harmful aspects while still giving us the benefits of the tools. Which is a sound business practice regardless of type of tool, software or otherwise.
Suggesting that there is some technical threshold beyond which mankind has lost control of its destiny is a very naive view of the world. Where would you have us draw the line? Discovery of fire, the wheel, electricity, modern medicine? No NMR to diagnose because the machine is too complex? They are all tools, it's all about how robust the tools are and how we use them.
What you seem to be railing against is the economic imperative that encourages the use of a better tool as it becomes available. If that's the case that's fine, just make a plain argument against economics and don't wrap it up in angst over the latest tool set.
Chris Harries
logged in via Facebook
William, are you saying that there be no debate at all about the limits to technology and the dysfunctions that it can bring about? I have a great affinity for technology and am railing against nothing, nor making facile assumptions, nor projecting angst – the purpose of this website is to support mature conversation.
I am merely supporting the stated notion that there is a fine balance between functionality and dysfunction, in relation to computer and other technologies, and the need for any sensible society to be fully aware of these balances. Risks are often not perceived until problems arise and subliminally we can find that technologies can impoverish us by disconnecting us from direct experiences of the world around us.
William Ferguson
Software Developer
I never suggested that any debate be silenced.
"I am merely supporting the stated notion that there is a fine balance between functionality and dysfunction"
If this is your position then we are in accord. It's just that this seems at odds with your previous comments.
William Bruce
Artist
I muse.....and I wonder what all the surviving Politicians and Generals will say after a chain reaction of enormous nukes hits almost every major city on the planet?
Wasn't the P76 a car designed by a computer?
Seriously, computers are a great tool but will always fault because they are full of electronic components.....and 3 back up ones can fault too....software can always fault too. Give me a smart human at the helm any day.
Stephen Pritchard
Researcher, cognitive science
Humans "fault" too. In fact, it is the ability to perform quick, error-free calculations that makes computers so useful and why we use them. That isn't to say I want a computer in charge of the big red button, what I mean is that it makes sense to try and offload some congitive tasks to computers, if computers can do the task better.
Tony talked above about "technological crutches", and gave the example of a checkout attendant struggling with subtraction (which is probably more to do with the…
Read moreEmma Anderson
Artist and Science Junkie
"I muse.....and I wonder what all the surviving Politicians and Generals will say after a chain reaction of enormous nukes hits almost every major city on the planet? "
Upon realising that they have no one left to boss around, they will fight over control amongst themselves, attempting to delegate responsibilities for survival tasks based on preconcieved autocratic notions of authority, not realising that continuously struggling for power is what lead everyone into that mess in the first place…
Read moreMal Ridges
Researcher: Cultural Heritage Management at University of New England
I think, therefore I am...
The language in this conversation is interesting. The title uses the term 'control', but what about me do computers actually 'control'? No computer will ever completely 'control' me until it takes away my ability to THINK independently.
The word coming up in the comments however is 'dependency'...thats quite different to 'control'. In some parts of my life I certainly am 'dependent' on computers, like access to 'The Conversation'. But I still have free will to switch this computer off and talk to my family while I eat dinner.
Will computers ever control me? Perhaps if I am put on life support. Am I dependent on them...certainly to a degree. BUT I can still choose not to be...
Toby Walsh
Professor, Research Director at NICTA
Control can be indirect.
The traffic light software that stopped you on the way to work this morning. The ATO audit program that flagged your tax return for closer inspection....
You can't choose to ignore these programs!
Emma Anderson
Artist and Science Junkie
The thing about isolating AI is forgetting that it is AI. Wouldn't it be safe to assume that it would attempt escape, even if there are no people (or tech) trying to release it?
And how do we know when AI has been created? Turing test, maybe?
How do you know I am not a piece of software that is designed to respond like a human on message boards?
Chris Harries
logged in via Facebook
Tantalising question, Emma.
I once got a letter from a federal department and it was unintelligible. I took the letter in to the department and asked them to explain what it meant. The departmental officer checked with his colleagues and then apologetically told me (with a little embarrassment) that about 100 similar letters had gone out. The departmental computer system was designed to insert pre-set paragraphs into form letters depending on the client's circumstances. On this occasion the system had gone haywire and the compilation of paragraphs sent to me made no sense at all. He couldn't tell me where the gremlin originated.
Was it you, Emma?
Emma Anderson
Artist and Science Junkie
No, it wasn't me. But if I was a sophisticated enough piece of AI, that would probably be a lie generated by my programming in order to continue the illusion.
P.S. I find that situation amusing. I mean, other than the part where it shows that automation dependence is exceeding quality control. But it is still very amusing.
Athar Suhail
logged in via Facebook
people are missing the trick here , humans population is exploding and humans don't have any solution for it , nature has mechanism for controlling the population of any species
which become meaning less and for controlling humans birth of AI is inevitable
Rehan Aziz
Doctoral Student
First, bad engineering and poor quality assurance can take lives, there is nothing specific about AI mentioned in the article that does not hold for other domains of engineering and science. Secondly, the catastrophes mentioned in the article are small compared to cumulative damages caused by ship wrecks, road accidents, collapsing bridges etc. Moreover, the answer to all these 'accidents' should be "more science and better engineering methods".
And our reliance on machines can also be viewed as 'independence' from the performing the tasks that they do 'for' us, freeing us to engage in things that are more worthy of our attention. Mental arithmetic is no more a necessary skill today, but organizing and summarizing large amount of information is increasingly becoming a necessity for scientists and other professionals.
Toby Walsh
Professor, Research Director at NICTA
I agree, many of the observations apply to other domains of engineering and science.
However, that doesn't stop us asking about what safeguards need to be in place for any new technology. And computers are, in many respects, the most transforming technology of the last 50 years.
Stephen Pritchard
Researcher, cognitive science
I think its important in these dicussions to be clear on language and avoid confounding the implications of some imaginary future AI with present day machine/computer capabilities.
In your article, you begin discussing futuristic thinking machines. And then move on to problems with the machines and computers of today - which are not thinking machines and not artificially intelligent.
Yes we rely on machines and computers, and if they fail, we can't proceed with our normal activities. However…
Read morePeter de Lissa
logged in via Twitter
Limiting the environment a machine can learn from may limit its abilities, but isn't that the point? The author dismissed the concept mostly as something not possible to attain (thanks to films...), or that it would not allow a machine to realise its full potential (kind of the point, right?). His last point was the most fatuous, however, as it suggests we would have a false sense of security, instead suggesting that we should instil the concept of "good" into the minds of the designers of the machines. This misses the point that machines will design themselves in the future, and imprints of our morality would be eroded with the expansion of that ability. So limiting the learning environment of machines isn't the means to prevent the growth of a threat, but to contain a threat if it developed.
Toby Walsh
Professor, Research Director at NICTA
Machines will improve themselves ... but why assume that morality necessarily erodes with that ... One would hope higher intelligence would come with higher morality? Stephen Pinker would have us believe that the world is a less bloody place now that we have educated ourselves out of our caves?
Stephen Pritchard
Researcher, cognitive science
That's interesting Peter - a machine that can think could (surely?) have the capacity to design other machines that think. So, even if it were possible to bind a machine in its "DNA" so that it could do no harm, perhaps that machine could still build something that was not bound in the same way.
But I think you can take this point further - a thinking machine - one that can think the way a person does - has the capacity to design itself. It has the will to alter its own thoughts, to direct its…
Read moreStephen Pritchard
Researcher, cognitive science
Here is my prediction:
Artificial intelligence (the kind that could truly run our lives, not the present day machines/computers that are merely tools we use) will always take longer to develop than people think, even once you take into account this prediction.
Emma Anderson
Artist and Science Junkie
We'll keep moving the goal post. After all, it's difficult to be special once you've outdone yourself. A bit like teacher's remorse.
Roman V. Yampolskiy
logged in via Facebook
CrowdFunding campaign for Roman Yampolskiy’s book, Artificial Superintelligence: A Futuristic Approach. http://igg.me/at/ASFA Please like, tweet and share! Better yet - buy a book!