tag:theconversation.com,2011:/us/topics/go-24328/articlesGo – The Conversation2020-12-02T13:28:54Ztag:theconversation.com,2011:article/1511812020-12-02T13:28:54Z2020-12-02T13:28:54ZAI makes huge progress predicting how proteins fold – one of biology’s greatest challenges – promising rapid drug development<figure><img src="https://images.theconversation.com/files/372322/original/file-20201201-15-s2hltf.png?ixlib=rb-1.1.0&rect=5%2C2%2C973%2C431&q=45&auto=format&w=496&fit=clip" /><figcaption><span class="caption">A simple chain of amino acids folds into a complex three-dimensional structure.</span> </figcaption></figure><p><strong>Takeaways</strong></p>
<ul>
<li><p><strong>A “deep learning” software program from Google-owned lab DeepMind showed great progress in solving one of biology’s greatest challenges – understanding protein folding.</strong> </p></li>
<li><p><strong>Protein folding is the process by which a protein takes its shape from a string of building blocks to its final three-dimensional structure, which determines its function.</strong></p></li>
<li><p><strong>By better predicting how proteins take their structure, or “fold,” scientists can more quickly develop drugs that, for example, block the action of crucial viral proteins.</strong> </p></li>
</ul>
<hr>
<p>Solving what biologists call “the protein-folding problem” is a big deal. Proteins are the workhorses of cells and are present in all living organisms. They are made up of long chains of amino acids and are vital for the structure of cells and communication between them as well as regulating all of the chemistry in the body. </p>
<p>This week, the Google-owned artificial intelligence company <a href="https://www.deepmind.com">DeepMind</a> demonstrated a deep-learning program called <a href="https://deepmind.com/blog/article/alphafold-a-solution-to-a-50-year-old-grand-challenge-in-biology">AlphaFold2</a>, which experts are calling a <a href="https://www.nature.com/articles/d41586-020-03348-4">breakthrough</a> toward solving the grand challenge of <a href="https://doi.org/10.1038/d41586-020-03348-4">protein folding</a>. </p>
<p>Proteins are long chains of amino acids linked together like beads on a string. But for a protein to do its job in the cell, it must “fold” – a process of twisting and bending that transforms the molecule into a complex three-dimensional structure that can interact with its target in the cell. If the folding is disrupted, then the protein won’t form the correct shape – and it won’t be able to perform its job inside the body. This can lead to disease – as is the case in a common disease like Alzheimer’s, and rare ones like cystic fibrosis.</p>
<p>Deep learning is a computational technique that uses the often hidden information contained in vast datasets to solve questions of interest. It’s been used widely in fields such as games, speech and voice recognition, autonomous cars, science and medicine.</p>
<p>I believe that tools like AlphaFold2 will help scientists to design new types of proteins, ones that may, for example, help break down plastics and fight future viral pandemics and disease. </p>
<p><a href="https://scholar.google.com/citations?user=RpiSPiwAAAAJ&hl=en">I am a computational chemist</a> and author of the book <a href="https://rowman.com/ISBN/9781633886407/The-State-of-Science-What-the-Future-Holds-and-the-Scientists-Making-It-Happen">The State of Science</a>. My students and I study the structure and properties of <a href="https://www.conncoll.edu/ccacad/zimmer/GFP-ww/GFP-1.htm">fluorescent proteins</a> using protein-folding computer programs based on classical physics. </p>
<p>After decades of study by thousands of research groups, these protein-folding prediction programs are very good at calculating structural changes that occur when we make small alterations to known molecules. </p>
<p>But they haven’t adequately managed to predict how proteins fold from scratch. Before deep learning came along, the protein-folding problem seemed impossibly hard, and it seemed poised to frustrate computational chemists for many decades to come.</p>
<figure class="align-center zoomable">
<a href="https://images.theconversation.com/files/372313/original/file-20201201-23-12msmry.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=1000&fit=clip"><img alt="" src="https://images.theconversation.com/files/372313/original/file-20201201-23-12msmry.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=754&fit=clip" srcset="https://images.theconversation.com/files/372313/original/file-20201201-23-12msmry.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=600&h=510&fit=crop&dpr=1 600w, https://images.theconversation.com/files/372313/original/file-20201201-23-12msmry.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=600&h=510&fit=crop&dpr=2 1200w, https://images.theconversation.com/files/372313/original/file-20201201-23-12msmry.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=600&h=510&fit=crop&dpr=3 1800w, https://images.theconversation.com/files/372313/original/file-20201201-23-12msmry.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=754&h=641&fit=crop&dpr=1 754w, https://images.theconversation.com/files/372313/original/file-20201201-23-12msmry.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=754&h=641&fit=crop&dpr=2 1508w, https://images.theconversation.com/files/372313/original/file-20201201-23-12msmry.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=754&h=641&fit=crop&dpr=3 2262w" sizes="(min-width: 1466px) 754px, (max-width: 599px) 100vw, (min-width: 600px) 600px, 237px"></a>
<figcaption>
<span class="caption">A chain of amino acids goes through several folding steps, which occurs through hydrogen bonds between amino acids in different regions of the protein, before arriving at the final structure. The example shown here is hemoglobin, a protein in red blood cells that transports oxygen to body tissues.</span>
<span class="attribution"><a class="source" href="https://upload.wikimedia.org/wikipedia/commons/2/26/225_Peptide_Bond-01.jpg">Anatomy & Physiology, Connexions website</a>, <a class="license" href="http://creativecommons.org/licenses/by/4.0/">CC BY</a></span>
</figcaption>
</figure>
<h2>Protein folding</h2>
<p>The sequence of the amino acids – which is encoded in DNA – defines the protein’s 3D shape. The shape determines its function. If the structure of the protein changes, it is unable to perform its function. Correctly predicting protein folds based on the amino acid sequence could revolutionize drug design, and explain the causes of new and old diseases. </p>
<p>All proteins with the same sequence of amino acid building blocks fold into the same three-dimensional form, which optimizes the interactions between the amino acids. They do this within milliseconds, although they have an astronomical number of possible configurations available to them – <a href="https://web.archive.org/web/20110523080407/http://www-miller.ch.cam.ac.uk/levinthal/levinthal.html">about 10 to the power of 300</a>. This massive number is what makes it hard to predict how a protein folds even when scientists know the full sequence of amino acids that go into making it. Previously predicting the structure of protein from the amino acid sequence was impossible. Protein structures were experimentally determined, a time-consuming and expensive endeavor. </p>
<p>Once researchers can better predict how proteins fold, they’ll be able to better understand how cells function and how misfolded proteins cause disease. Better protein prediction tools will also help us design drugs that can target a particular topological region of a protein where chemical reactions take place. </p>
<figure class="align-center ">
<img alt="" src="https://images.theconversation.com/files/372314/original/file-20201201-23-86jeuv.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=754&fit=clip" srcset="https://images.theconversation.com/files/372314/original/file-20201201-23-86jeuv.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=600&h=300&fit=crop&dpr=1 600w, https://images.theconversation.com/files/372314/original/file-20201201-23-86jeuv.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=600&h=300&fit=crop&dpr=2 1200w, https://images.theconversation.com/files/372314/original/file-20201201-23-86jeuv.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=600&h=300&fit=crop&dpr=3 1800w, https://images.theconversation.com/files/372314/original/file-20201201-23-86jeuv.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=754&h=377&fit=crop&dpr=1 754w, https://images.theconversation.com/files/372314/original/file-20201201-23-86jeuv.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=754&h=377&fit=crop&dpr=2 1508w, https://images.theconversation.com/files/372314/original/file-20201201-23-86jeuv.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=754&h=377&fit=crop&dpr=3 2262w" sizes="(min-width: 1466px) 754px, (max-width: 599px) 100vw, (min-width: 600px) 600px, 237px">
<figcaption>
<span class="caption">What’s your move?</span>
<span class="attribution"><a class="source" href="https://www.gettyimages.com/detail/photo/robot-hand-chessboard-royalty-free-image/1255171787?adppopup=true">style-photography/Getty Images</a></span>
</figcaption>
</figure>
<h2>AlphaFold is born from deep-learning chess, Go and poker games</h2>
<p>The success of DeepMind’s protein-folding prediction program, called <a href="https://deepmind.com/research/case-studies/alphafold">AlphaFold</a>, is not unexpected. Other deep-learning programs written by <a href="https://deepmind.com/about">DeepMind</a> have demolished the world’s best chess, Go and poker players.</p>
<p>In 2016 <a href="https://www.chessprogramming.org/Stockfish">Stockfish-8</a>, an open-source chess engine, was the world’s computer chess champion. It evaluated 70 million chess positions per second and had centuries of accumulated human chess strategies and decades of computer experience to draw upon. It played efficiently and brutally, mercilessly beating all its human challengers without an ounce of finesse. Enter deep learning. </p>
<p>On Dec. 7, 2017, Google’s deep-learning chess program <a href="http://doi.org/10.1126/science.aar6404">AlphaZero</a> thrashed Stockfish-8. The chess engines played 100 games, with AlphaZero winning 28 and tying 72. It didn’t lose a single game. AlphaZero did only 80,000 calculations per second, as opposed to Stockfish-8’s 70 million calculations, and it took just four hours to learn chess from scratch by playing against itself a few million times and optimizing its neural networks as it learned from its experience. </p>
<p><a href="https://web.stanford.edu/%7Esurag/posts/alphazero.html">AlphaZero</a> didn’t learn anything from humans or chess games played by humans. It taught itself and, in the process, derived strategies never seen before. In a <a href="https://doi.org/10.1126/science.aaw2221">commentary</a> in Science magazine, former world chess champion Garry Kasparov wrote that by learning from playing itself, AlphaZero developed strategies that “reflect the truth” of chess rather than reflecting “the priorities and prejudices” of the programmers. “It’s the embodiment of the cliché ‘work smarter, not harder.’” </p>
<figure>
<iframe width="440" height="260" src="https://www.youtube.com/embed/gg7WjuFs8F4?wmode=transparent&start=0" frameborder="0" allowfullscreen=""></iframe>
<figcaption><span class="caption">How do proteins fold?</span></figcaption>
</figure>
<h2>CASP – the Olympics for molecular modelers</h2>
<p>Every two years, the world’s top computational chemists test the abilities of their programs to predict the folding of proteins and compete in the <a href="https://predictioncenter.org">Critical Assessment of Structure Prediction</a> (CASP) competition. </p>
<p>In the competition, teams are given the linear sequence of amino acids for about 100 proteins for which the 3D shape is known but hasn’t yet been published; they then have to compute how these sequences would fold. In 2018 AlphaFold, the deep-learning rookie at the competition, beat all the traditional programs – but barely. </p>
<p>Two years later, on Monday, it was announced that Alphafold2 had won the 2020 competition by a healthy margin. It whipped its competitors, and its predictions were comparable to the existing experimental results determined through gold standard techniques like X-ray diffraction crystallography and cryo-electron microscopy. Soon I expect AlphaFold2 and its progeny will be the methods of choice to determine protein structures before resorting to experimental techniques that require painstaking, laborious work on expensive instrumentation.</p>
<p>One of the reasons for AlphaFold2’s success is that it could use the <a href="https://www.rcsb.org/">Protein Database</a>, which has over 170,000 experimentally determined 3D structures, to train itself to calculate the correctly folded structures of proteins. </p>
<p>The potential impact of AlphaFold can be appreciated if one compares the number of all published protein structures – approximately 170,000 – with the 180 million DNA and protein sequences deposited in the <a href="https://www.uniprot.org">Universal Protein Database</a>. AlphaFold will help us sort through treasure troves of DNA sequences hunting for new proteins with unique structures and <a href="https://deepmind.com/blog/article/alphafold-a-solution-to-a-50-year-old-grand-challenge-in-biology">functions</a>.</p>
<h2>Has AlphaFold made me, a molecular modeler, redundant?</h2>
<p>As with the chess and Go programs – AlphaZero and AlphaGo – we don’t exactly know what the AlphaFold2 algorithm is doing and why it uses certain correlations, but we do know that it works. </p>
<p>Besides helping us predict the structures of important proteins, understanding AlphaFold’s “thinking” will also help us gain new insights into the mechanism of protein folding.</p>
<p>[<em>Deep knowledge, daily.</em> <a href="https://theconversation.com/us/newsletters/the-daily-3?utm_source=TCUS&utm_medium=inline-link&utm_campaign=newsletter-text&utm_content=deepknowledge">Sign up for The Conversation’s newsletter</a>.]</p>
<p>One of the most common fears expressed about AI is that it will lead to large-scale unemployment. AlphaFold still has a significant way to go before it can consistently and successfully predict protein folding. </p>
<p>However, once it has matured and the program can simulate protein folding, computational chemists will be integrally involved in improving the programs, trying to understand the underlying correlations used, and applying the program to solve important problems such as the protein misfolding associated with many diseases such as Alzheimer’s, Parkinson’s, cystic fibrosis and Huntington’s disease. </p>
<p>AlphaFold and its offspring will certainly change the way computational chemists work, but it won’t make them redundant. Other areas won’t be as fortunate. In the past robots were able to replace humans doing manual labor; with AI, our cognitive skills are also being challenged.</p><img src="https://counter.theconversation.com/content/151181/count.gif" alt="The Conversation" width="1" height="1" />
<p class="fine-print"><em><span>Marc Zimmer does not work for, consult, own shares in or receive funding from any company or organization that would benefit from this article, and has disclosed no relevant affiliations beyond their academic appointment.</span></em></p>Scientists in an artificial intelligence lab have made a breakthrough in solving the problem of how proteins fold into their final three-dimensional shape. The work could speed up creation of drugs.Marc Zimmer, Professor of Chemistry, Connecticut CollegeLicensed as Creative Commons – attribution, no derivatives.tag:theconversation.com,2011:article/944372018-04-05T14:44:14Z2018-04-05T14:44:14ZTo drive AI forward, teach computers to play old-school text adventure games<figure><img src="https://images.theconversation.com/files/214305/original/file-20180411-543-1dcho2z.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=496&fit=clip" /><figcaption><span class="caption">Ready player one?</span> <span class="attribution"><a class="source" href="https://commons.wikimedia.org/wiki/File:Colossal_Cave_Adventure_on_VT100_terminal.jpg#/media/File:Colossal_Cave_Adventure_on_VT100_terminal.jpg">Wikimedia</a></span></figcaption></figure><p>Games have long been used as test beds and benchmarks for artificial intelligence, and there has been no shortage of achievements in recent months. Google DeepMind’s <a href="https://theconversation.com/googles-latest-go-victory-shows-machines-are-no-longer-just-learning-theyre-teaching-78410">AlphaGo</a> and <a href="https://www.theregister.co.uk/2017/12/19/poker_bot_libratus_ai/">poker bot Libratus</a> from Carnegie Mellon University have both beaten human experts at games that have traditionally been hard for AI – some 20 years after IBM’s DeepBlue achieved the same feat <a href="https://www.theguardian.com/theguardian/2011/may/12/deep-blue-beats-kasparov-1997">in chess</a>. </p>
<p>Games like these have the attraction of clearly defined rules; they are relatively simple and cheap for AI researchers to work with, and they provide a variety of cognitive challenges at any desired level of difficulty. By inventing algorithms that play them well, researchers hope to gain insights into the mechanisms needed to function autonomously. </p>
<p>With the arrival of the latest techniques in AI and machine learning, attention is <a href="https://project.dke.maastrichtuniversity.nl/cig2018/?page_id=255">now shifting</a> to visually detailed computer games – including the 3D shooter Doom, <a href="https://github.com/mgbellemare/Arcade-Learning-Environment">various 2D Atari games</a> such as Pong and Space Invaders, and the real-time strategy game StarCraft. </p>
<p>This is all certainly progress, but a key part of the bigger AI picture is being overlooked. Research has prioritised games in which all the actions that can be performed are known in advance, be it moving a knight or firing a weapon. The computer is given all the options from the outset and the focus is on how well it chooses between them. The problem is that this disconnects AI research from the task of making computers genuinely autonomous. </p>
<h2>Banana skins</h2>
<p>Getting computers to determine which actions even exist in a given context presents conceptual and practical challenges which games researchers have barely attempted to resolve so far. The “monkey and bananas” problem is one example of a longstanding AI conundrum in which no recent progress has been made. </p>
<figure class="align-right zoomable">
<a href="https://images.theconversation.com/files/213170/original/file-20180404-189807-zzpsqv.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=1000&fit=clip"><img alt="" src="https://images.theconversation.com/files/213170/original/file-20180404-189807-zzpsqv.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=237&fit=clip" srcset="https://images.theconversation.com/files/213170/original/file-20180404-189807-zzpsqv.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=600&h=690&fit=crop&dpr=1 600w, https://images.theconversation.com/files/213170/original/file-20180404-189807-zzpsqv.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=600&h=690&fit=crop&dpr=2 1200w, https://images.theconversation.com/files/213170/original/file-20180404-189807-zzpsqv.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=600&h=690&fit=crop&dpr=3 1800w, https://images.theconversation.com/files/213170/original/file-20180404-189807-zzpsqv.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=754&h=867&fit=crop&dpr=1 754w, https://images.theconversation.com/files/213170/original/file-20180404-189807-zzpsqv.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=754&h=867&fit=crop&dpr=2 1508w, https://images.theconversation.com/files/213170/original/file-20180404-189807-zzpsqv.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=754&h=867&fit=crop&dpr=3 2262w" sizes="(min-width: 1466px) 754px, (max-width: 599px) 100vw, (min-width: 600px) 600px, 237px"></a>
<figcaption>
<span class="caption">Headscratcher.</span>
<span class="attribution"><a class="source" href="https://www.shutterstock.com/image-photo/monkey-man-holding-banana-over-colorful-283183991?src=UaQrgHzv4Gm6OcQmcC2fzA-1-42">Luis Molinero</a></span>
</figcaption>
</figure>
<p>The problem was <a href="https://www.sciencedirect.com/science/article/pii/S0004370210001827">originally posed</a> by John McCarthy, one of the founding fathers of AI, in 1963: there is a room containing a chair, a stick, a monkey and a bunch of bananas hanging on a ceiling hook. The task is for a computer to come up with a sequence of actions to enable the monkey to acquire the bananas. </p>
<p>McCarthy made a key distinction between two aspects of this task in terms of artificial intelligence. Physical feasibility – determining whether a particular sequence of actions is physically realisable; and epistemic or knowledge-related feasibility – determining which possible actions for the monkey actually exist. </p>
<p>Determining what is physically feasible for the monkey is very easy for a computer if it is told all the possible actions in advance – “climb on chair”, “wave stick” and so forth. A simple program that instructs the computer to go through all the possible sequences of actions one by one will quickly arrive at the best solution. </p>
<p>If the computer has to first determine which actions are even possible, however, it is a much tougher challenge. It raises questions about how we represent knowledge, the necessary and sufficient conditions of knowing something, and how we know when enough knowledge has been acquired. In highlighting these problems, McCarthy <a href="https://dl.acm.org/citation.cfm?id=216000">said</a>:</p>
<blockquote>
<p>Our ultimate objective is to make programs that learn from their experience as effectively as humans do.</p>
</blockquote>
<p>Until computers can tackle problems without any predetermined description of possible actions, this objective can’t be achieved. It is unfortunate that AI researchers are neglecting this: not only are these problems harder and more interesting, they look like a prerequisite for making further meaningful progress in the field. </p>
<h2>Text appeal</h2>
<p>To operate autonomously in a complex environment, it is impossible to describe in advance how best to manipulate – or even characterise – the objects there. Teaching computers to get around these difficulties immediately leads to deep questions about learning from previous experience.</p>
<p>Rather than focusing on games like Doom or StarCraft, where it is possible to avoid this problem, a more promising test for modern AI could be the humble text adventure from the 1970s and 1980s. </p>
<p>In the days before computers had sophisticated graphics capabilities, games like Colossal Cave and Zork were popular. Players were told about their environment by messages on the screen:</p>
<figure class="align-center zoomable">
<a href="https://images.theconversation.com/files/213173/original/file-20180404-189816-e49be.png?ixlib=rb-1.1.0&q=45&auto=format&w=1000&fit=clip"><img alt="" src="https://images.theconversation.com/files/213173/original/file-20180404-189816-e49be.png?ixlib=rb-1.1.0&q=45&auto=format&w=754&fit=clip" srcset="https://images.theconversation.com/files/213173/original/file-20180404-189816-e49be.png?ixlib=rb-1.1.0&q=45&auto=format&w=600&h=376&fit=crop&dpr=1 600w, https://images.theconversation.com/files/213173/original/file-20180404-189816-e49be.png?ixlib=rb-1.1.0&q=30&auto=format&w=600&h=376&fit=crop&dpr=2 1200w, https://images.theconversation.com/files/213173/original/file-20180404-189816-e49be.png?ixlib=rb-1.1.0&q=15&auto=format&w=600&h=376&fit=crop&dpr=3 1800w, https://images.theconversation.com/files/213173/original/file-20180404-189816-e49be.png?ixlib=rb-1.1.0&q=45&auto=format&w=754&h=473&fit=crop&dpr=1 754w, https://images.theconversation.com/files/213173/original/file-20180404-189816-e49be.png?ixlib=rb-1.1.0&q=30&auto=format&w=754&h=473&fit=crop&dpr=2 1508w, https://images.theconversation.com/files/213173/original/file-20180404-189816-e49be.png?ixlib=rb-1.1.0&q=15&auto=format&w=754&h=473&fit=crop&dpr=3 2262w" sizes="(min-width: 1466px) 754px, (max-width: 599px) 100vw, (min-width: 600px) 600px, 237px"></a>
<figcaption>
<span class="caption">Picture this.</span>
</figcaption>
</figure>
<p>They had to respond with simple instructions, usually in the form of a verb or a verb plus a noun – “look”, “take box” and so on. Part of the challenge was to work out which actions were possible and useful and to respond accordingly. </p>
<p>A good challenge for modern AI would be to take on the role of a player in such an adventure. The computer would have to make sense of the text descriptions on the screen and respond to them with actions, using some predictive mechanism to determine their likely effect. </p>
<p>More sophisticated behaviours on part of the computer would involve exploring the environment, defining goals, making goal-oriented action choices and solving the various intellectual challenges typically required to progress. </p>
<p>How well modern AI methods of the kind promoted by tech giants like IBM, Google, Facebook or Microsoft would fare in these text adventures is an open question – as is how much specialist human knowledge they would require for each new scenario. </p>
<p>To measure progress in this area, for the past two years we <a href="http://atkrye.github.io/IEEE-CIG-Text-Adventurer-Competition/2018/01/16/announceThirdYear/">have been running a competition</a> at the IEEE Conference on Computational Intelligence and Games, which <a href="https://project.dke.maastrichtuniversity.nl/cig2018/">this year takes place in Maastricht</a> in the Netherlands in August. Competitors submit entries in advance, and can use the AI technology of their choice to build programs that can play these games by making sense of a text description and outputting appropriate text commands in return. </p>
<p>In short, researchers need to reconsider their priorities if AI is to keep progressing. If unearthing the discipline’s neglected roots turns out to be fruitful, the monkey may finally gets his bananas after all.</p><img src="https://counter.theconversation.com/content/94437/count.gif" alt="The Conversation" width="1" height="1" />
<p class="fine-print"><em><span>The authors do not work for, consult, own shares in or receive funding from any company or organisation that would benefit from this article, and have disclosed no relevant affiliations beyond their academic appointment.</span></em></p>It’s time programmers looked out old computer text adventures like Zork and Colossal Cave from the 1970s and 1980s.Jerry Swan, Senior Research Fellow, University of YorkHendrik Baier, Research Associate for Artificial Intelligence and Data Analytics, University of YorkTimothy Atkinson, Doctoral Researcher, University of YorkLicensed as Creative Commons – attribution, no derivatives.tag:theconversation.com,2011:article/859792017-10-20T06:05:24Z2017-10-20T06:05:24ZGoogle’s new Go-playing AI learns fast, and even thrashed its former self<figure><img src="https://images.theconversation.com/files/191163/original/file-20171020-27065-1r5j84a.jpg?ixlib=rb-1.1.0&rect=0%2C670%2C6390%2C4119&q=45&auto=format&w=496&fit=clip" /><figcaption><span class="caption">Better than human: the artificial intelligence that learned to master Go in just three days.</span> <span class="attribution"><span class="source">Shutterstock/maxuser</span></span></figcaption></figure><p>Just last year Google DeepMind’s <a href="https://deepmind.com/research/alphago/">AlphaGo</a> took the world of Artificial Intelligence (AI) by storm, showing that a <a href="https://theconversation.com/googles-go-victory-shows-ai-thinking-can-be-unpredictable-and-thats-a-concern-56209">computer program could beat the world’s best human Go players</a>. </p>
<p>But in a demonstration of the feverish rate of progress in modern AI, details of a new milestone reached by an improved version called <a href="https://deepmind.com/blog/alphago-zero-learning-scratch/">AlphaGo Zero</a> were <a href="https://www.nature.com/nature/journal/v550/n7676/full/nature24270.html">published this week in Nature</a>.</p>
<p>Using less computing power and only three days of training time, AlphaGo Zero beat the original AlphaGo in a 100-game match by 100 to 0. It wasn’t even worth humans showing up.</p>
<hr>
<p>
<em>
<strong>
Read more:
<a href="https://theconversation.com/why-google-wants-to-think-more-like-you-and-less-like-a-machine-79911">Why Google wants to think more like you and less like a machine</a>
</strong>
</em>
</p>
<hr>
<h2>Learning to play Go</h2>
<p>Go is a game of strategy between two players who take it in turns to place “stones” on a 19x19 board. The goal is to surround a larger area of the board than your opponent.</p>
<figure class="align-center zoomable">
<a href="https://images.theconversation.com/files/191164/original/file-20171020-28465-1ljcwbo.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=1000&fit=clip"><img alt="" src="https://images.theconversation.com/files/191164/original/file-20171020-28465-1ljcwbo.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=754&fit=clip" srcset="https://images.theconversation.com/files/191164/original/file-20171020-28465-1ljcwbo.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=600&h=400&fit=crop&dpr=1 600w, https://images.theconversation.com/files/191164/original/file-20171020-28465-1ljcwbo.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=600&h=400&fit=crop&dpr=2 1200w, https://images.theconversation.com/files/191164/original/file-20171020-28465-1ljcwbo.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=600&h=400&fit=crop&dpr=3 1800w, https://images.theconversation.com/files/191164/original/file-20171020-28465-1ljcwbo.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=754&h=503&fit=crop&dpr=1 754w, https://images.theconversation.com/files/191164/original/file-20171020-28465-1ljcwbo.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=754&h=503&fit=crop&dpr=2 1508w, https://images.theconversation.com/files/191164/original/file-20171020-28465-1ljcwbo.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=754&h=503&fit=crop&dpr=3 2262w" sizes="(min-width: 1466px) 754px, (max-width: 599px) 100vw, (min-width: 600px) 600px, 237px"></a>
<figcaption>
<span class="caption">The game of Go, simple to learn but a lifetime to master… for a human.</span>
<span class="attribution"><span class="source">Paragorn Dangsombroon/Shutterstock</span></span>
</figcaption>
</figure>
<p>Go has proved much more challenging than chess for computers to master. There are many more possible moves in each position in Go than chess, and many more possible games.</p>
<p>The <a href="https://blog.google/topics/machine-learning/alphago-machine-learning-game-go/">original AlphaGo first learned</a> from studying 30 million moves of expert human play. It then improved beyond human expertise by playing many games against itself, taking several months of computer time.</p>
<p>By contrast, AlphaGo Zero never saw humans play. Instead, it began by knowing only the rules of the game. From a relatively modest five million games of self-play, taking only three days on a smaller computer than the original AlphaGo, it then learned super-AlphaGo performance.</p>
<p>Fascinatingly, its learning roughly mimicked some of the stages through which humans progress as they master Go. AlphaGo Zero rapidly learned to reject naively short-term goals and developed more strategic thinking, generating many of the patterns of moves often used by top-level human experts. </p>
<p>But remarkably it then started rejecting some of these patterns in favour of new strategies never seen before in human play.</p>
<h2>Beyond human play</h2>
<p>AlphaGo Zero achieved this feat by approaching the problem differently from the original AlphaGo. Both versions use a combination of two of the most powerful algorithms currently fuelling AI: <a href="https://theconversation.com/no-more-playing-games-alphago-ai-to-tackle-some-real-world-challenges-78472">deep learning and reinforcement learning</a>.</p>
<p>To play a game like Go, there are two basic things the program needs to learn. The first is a policy: the probability of making each of the possible moves in a given position. The second is a value: the probability of winning from any given position.</p>
<p>In the pure reinforcement learning approach of AlphaGo Zero, the only information available to learn policies and values was for it to predict who might ultimately win. To make this prediction it used its current policy and values, but at the start these were random.</p>
<p>This is clearly a more challenging approach than the original AlphaGo, which used expert human moves to get a head-start on learning. But the earlier version learned policies and values with separate neural networks. </p>
<p>The algorithmic breakthrough in AlphaGo Zero was to figure out how these could be combined in just one network. This allowed the process of training by self-play to be greatly simplified, and made it feasible to start from a clean slate rather than first learning what expert humans would do.</p>
<figure>
<iframe width="440" height="260" src="https://www.youtube.com/embed/tXlM99xPQC8?wmode=transparent&start=0" frameborder="0" allowfullscreen=""></iframe>
<figcaption><span class="caption">How AlphaGo Zero learned to master Go.</span></figcaption>
</figure>
<p>An Elo rating is a widely used measure of the performance of players in games such as Go and chess. The best human player so far, <a href="https://www.goratings.org/en/players/1195.html">Ke Jie</a>, currently has an Elo rating of about 3,700.</p>
<p>AlphaGo Zero trained for three days and achieved an Elo rating of more than 4,000, while an expanded version of the same algorithm trained for 40 days and achieved almost 5,200.</p>
<p>This is an astonishingly large step up from the best human – far bigger than the current gap between the best human chess player <a href="https://ratings.fide.com/top_files.phtml?id=1503014">Magnus Carlsen</a> (about 2,800) and <a href="http://www.computerchess.org.uk/ccrl/4040/">chess program</a> (about 3,400).</p>
<h2>The next challenge</h2>
<p>AlphaGo Zero is an important step forward for AI because it demonstrates the feasibility of pure reinforcement learning, uncorrupted by any human guidance. This removes the need for lots of expert human knowledge to get started, which in some domains can be hard to obtain. </p>
<p>It also means the algorithm is free to develop completely new approaches that might have been much harder to find had it been been initially constrained to “think inside the human box”. Remarkably, this strategy also turns out to be more computationally efficient.</p>
<p>But Go is a tightly constrained game of perfect information, without the messiness of most real-world problems. Training AlphaGo Zero required the accurate simulation of millions of games, following the rules of Go. </p>
<p>For many practical problems such simulations are computationally unfeasible, or the rules themselves are less clear. </p>
<hr>
<p>
<em>
<strong>
Read more:
<a href="https://theconversation.com/no-more-playing-games-alphago-ai-to-tackle-some-real-world-challenges-78472">No more playing games: AlphaGo AI to tackle some real world challenges</a>
</strong>
</em>
</p>
<hr>
<p>There are still many further problems to be solved to create a general-purpose AI, one that can tackle a wide range of practical problems without domain-specific human intervention. </p>
<p>But even though humans have now comprehensively lost the battle with Go algorithms, luckily AI (unlike Go) is not a zero-sum game. Many of AlphaGo Zero’s games <a href="http://www.alphago-games.com/">have now been published</a>, providing a lifetime of inspirational study for human Go players.</p>
<p>More importantly, AlphaGo Zero represents a step towards a world where humans can harness powerful AIs to help find unimaginably (to humans) creative solutions to difficult problems. In the world of AI, there has never been a better time to Go for it.</p><img src="https://counter.theconversation.com/content/85979/count.gif" alt="The Conversation" width="1" height="1" />
<p class="fine-print"><em><span>Geoff Goodhill does not work for, consult, own shares in or receive funding from any company or organisation that would benefit from this article, and has disclosed no relevant affiliations beyond their academic appointment.</span></em></p>The new AlphaGo Zero artificial intelligence took just days to learn to play Go from scratch, with no human intervention. It even learned strategies never seen before in human play.Geoff Goodhill, Professor of Neuroscience and Mathematics, The University of QueenslandLicensed as Creative Commons – attribution, no derivatives.tag:theconversation.com,2011:article/784722017-06-05T20:02:23Z2017-06-05T20:02:23ZNo more playing games: AlphaGo AI to tackle some real world challenges<p>Humankind lost another important battle with artificial intelligence (AI) last month, when <a href="https://deepmind.com/research/alphago/">AlphaGo</a> beat the world’s leading Go player Ke Jie by three games to zero.</p>
<p>AlphaGo is an AI program developed by <a href="https://deepmind.com/">DeepMind</a>, part of Google’s parent company <a href="https://abc.xyz/">Alphabet</a>. Last year it <a href="http://www.abc.net.au/news/2016-03-15/google-ai-alphago-gets-divine-go-ranking/7249256">beat another leading player</a>, Lee Se-dol, by four games to one, but since then AlphaGo has substantially improved.</p>
<p>Ke Jie described AlphaGo’s skill as “<a href="https://www.cnet.com/au/news/google-alphago-ai-artificial-intelligence-go-ke-jie/">like a God of Go</a>”. </p>
<p>AlphaGo will now <a href="https://deepmind.com/blog/alphagos-next-move/">retire from playing Go</a>, leaving behind a legacy of games played against itself. They’ve been described by one Go expert as like “<a href="https://deepmind.com/research/alphago/alphago-vs-alphago-self-play-games/">games from far in the future</a>”, which humans will study for years to improve their own play.</p>
<h2>Ready, set, Go</h2>
<p>Go is an ancient game that essentially pits two players – one playing black pieces the other white – for dominance on board usually marked with 19 horizontal and 19 vertical lines.</p>
<figure class="align-center zoomable">
<a href="https://images.theconversation.com/files/172208/original/file-20170605-31005-19hr6l0.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=1000&fit=clip"><img alt="" src="https://images.theconversation.com/files/172208/original/file-20170605-31005-19hr6l0.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=754&fit=clip" srcset="https://images.theconversation.com/files/172208/original/file-20170605-31005-19hr6l0.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=600&h=377&fit=crop&dpr=1 600w, https://images.theconversation.com/files/172208/original/file-20170605-31005-19hr6l0.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=600&h=377&fit=crop&dpr=2 1200w, https://images.theconversation.com/files/172208/original/file-20170605-31005-19hr6l0.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=600&h=377&fit=crop&dpr=3 1800w, https://images.theconversation.com/files/172208/original/file-20170605-31005-19hr6l0.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=754&h=473&fit=crop&dpr=1 754w, https://images.theconversation.com/files/172208/original/file-20170605-31005-19hr6l0.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=754&h=473&fit=crop&dpr=2 1508w, https://images.theconversation.com/files/172208/original/file-20170605-31005-19hr6l0.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=754&h=473&fit=crop&dpr=3 2262w" sizes="(min-width: 1466px) 754px, (max-width: 599px) 100vw, (min-width: 600px) 600px, 237px"></a>
<figcaption>
<span class="caption">A typical game of Go: simple to learn but a lifetime to master.</span>
<span class="attribution"><a class="source" href="https://www.flickr.com/photos/alper/30626352/">Flickr/Alper Cugun</a>, <a class="license" href="http://creativecommons.org/licenses/by/4.0/">CC BY</a></span>
</figcaption>
</figure>
<p>Go is a far more difficult game for computers to play than chess, because the number of possible moves in each position is much larger. This makes searching many moves ahead – feasible for computers in chess – very difficult in Go.</p>
<p>DeepMind’s breakthrough was the development of general-purpose learning algorithms that can, in principle, be trained in more societal-relevant domains than Go.</p>
<p>DeepMind says the research team behind AplhaGo is <a href="https://deepmind.com/blog/alphagos-next-move/">looking to pursue other complex problems</a>, such as finding new cures for diseases, dramatically reducing energy consumption or inventing revolutionary new materials. It adds:</p>
<blockquote>
<p>If AI systems prove they are able to unearth significant new knowledge and strategies in these domains too, the breakthroughs could be truly remarkable. We can’t wait to see what comes next.</p>
</blockquote>
<p>This does open up many opportunities for the future, but challenges still remain.</p>
<h2>Neuroscience meets AI</h2>
<p>AlphaGo combines the two most powerful ideas about learning to emerge from the past few decades: deep learning and reinforcement learning. Remarkably, both were originally inspired by how biological brains learn from experience.</p>
<p>In the human brain, sensory information is processed in a series of layers. For instance, visual information is first transformed in the retina, then in the midbrain, and then through many different areas of the cerebral cortex.</p>
<p>This creates a hierarchy of representations where simple, local features are extracted first, and then more complex, global features are built from these.</p>
<p>The AI equivalent is called deep learning; deep because it involves many layers of processing in simple neuron-like computing units.</p>
<p>But to survive in the world, animals need to not only recognise sensory information, but also act on it. Generations of scientists and psychologists have studied how animals learn to take a series of actions that maximise their reward. </p>
<p>This has led to mathematical theories of reinforcement learning that can now be implemented in AI systems. The most powerful of these is temporal difference learning, which improves actions by maximising its expectation of future reward.</p>
<h2>The best moves</h2>
<p>By combining deep learning and reinforcement learning in a series of artificial neural networks, AlphaGo first learned human expert-level play in Go from 30 million moves from human games.</p>
<p>But then it started playing against itself, using the outcome of each game to relentlessly refine its decisions about the best move in each board position. A value network learned to predict the likely outcome given any position, while a policy network learned the best action to take in each situation. </p>
<p>Although it couldn’t sample every possible board position, AlphaGo’s neural networks extracted key ideas about strategies that work well in any position. It is these countless hours of self-play that led to AlphaGo’s improvement over the past year.</p>
<p>Unfortunately, as yet there is no known way to interrogate the network to directly read out what these key ideas are. Instead we can only study its games and hope to learn from these. </p>
<p>This is one of the problems with using such neural network algorithms to help make decisions in, for instance, the legal system: they can’t explain their reasoning. </p>
<p>We still understand relatively little about how biological brains actually learn, and neuroscience will continue to provide new inspiration for improvements in AI. </p>
<p>Humans can learn to become expert Go players based on far less experience than AlphaGo needed to reach that level, so there is clearly room for further developing the algorithms.</p>
<p>Also much of AlphaGo’s power is based on a technique called back-propagation learning that helps it correct errors. But the relationship between this and learning in real brains is still unclear.</p>
<h2>What’s next?</h2>
<p>The game of Go provided a nicely constrained development platform for optimising these learning algorithms. But many real world problems are messier than this, and have less opportunity for the equivalent of self-play (for instance self-driving cars).</p>
<p>So are there problems to which the current algorithms can be fairly immediately applied?</p>
<p>One example may be optimisation in controlled industrial settings. Here the goal is often to complete a complex series of tasks while satisfying multiple constraints and minimising cost.</p>
<p>As long as the possibilities can be accurately simulated, these algorithms can explore and learn from a vastly larger space of outcomes than will ever be possible for humans. Thus DeepMind’s bold claims seem likely to be realised, and as the company says, we can’t wait to see what comes next.</p><img src="https://counter.theconversation.com/content/78472/count.gif" alt="The Conversation" width="1" height="1" />
<p class="fine-print"><em><span>Geoff Goodhill receives funding from the Australian Research Council and the National Health and Medical Research Council. </span></em></p>The artificial intelligence that beat a world master at the game of Go is now to be directed at more complex global problems. So what can we expect?Geoff Goodhill, Professor of Neuroscience and Mathematics, The University of QueenslandLicensed as Creative Commons – attribution, no derivatives.tag:theconversation.com,2011:article/784102017-05-26T13:05:17Z2017-05-26T13:05:17ZGoogle’s latest Go victory shows machines are no longer just learning, they’re teaching<p>Just over 20 years ago was the first time a <a href="https://theconversation.com/twenty-years-on-from-deep-blue-vs-kasparov-how-a-chess-match-started-the-big-data-revolution-76882">computer beat a human world champion</a> in a chess match, when IBM’s Deep Blue supercomputer beat Gary Kasparov in a narrow victory of 3½ games to 2½. Just under a decade later, machines were deemed to have conquered the game of chess when Deep Fritz, a piece of software running on a desktop PC, <a href="http://en.chessbase.com/post/kramnik-vs-deep-fritz-computer-wins-match-by-4-2">beat 2006 world champion Vladimir Kramnik</a>. Now the ability of computers to take on humanity has taken a step further by mastering the far more complex board game Go, with Google’s AlphaGo program <a href="https://www.nytimes.com/2017/05/25/business/google-alphago-defeats-go-ke-jie-again.html?_r=0">beating world number one</a> Ke Jie twice in a best-of-three series.</p>
<p>This signifcant milestone shows just how far computers have come in the past 20 years. DeepBlue’s victory at chess showed machines could rapidly process huge amounts of information, <a href="https://theconversation.com/twenty-years-on-from-deep-blue-vs-kasparov-how-a-chess-match-started-the-big-data-revolution-76882">paving the way for the big data revolution</a> we see today. But AlphaGo’s triumph represents the development of real artificial intelligence by a machine that can recognise patterns and learn the best way to respond to them. What’s more, it may signify a new evolution in AI, where computers not only learn how to beat us but can start to teach us as well.</p>
<p>Go is considered one of the <a href="https://www.quora.com/Is-Go-the-most-complicated-2-player-board-game">world’s most complex board games</a>. Like chess, it’s a game of strategy but it also has several key differences that make it much harder for a computer to play. The rules are relatively simple but the strategies involved to play the game are highly complex. It is also much harder to calculate the end position and winner in the game of Go. </p>
<p>It has a larger board (a 19x19 grid rather than an 8x8 one) and an unlimited number of pieces, so there are many more ways that the board can be arranged. Whereas chess pieces start in set positions and can each make a limited number of moves each turn, Go starts with a blank board and players can place a piece in any of the 361 free spaces. Each game takes on average twice as many turns as chess and there are six times as many legal move options per turn.</p>
<p>Each of these features means you can’t build a Go program using the same techniques as for chess machines. These tend to use a “brute force” approach of analysing the potential of large numbers of possible moves to select the best one. Feng-Hsiung Hsu, one of the key contributors to the DeepBlue team, argued in 2007 that <a href="http://spectrum.ieee.org/computing/software/cracking-go">applying this strategy to Go</a> would require a million-fold increase in processing speed over DeepBlue so a computer could analyse 100 trillion positions per second.</p>
<h2>Learning new moves</h2>
<p>The strategy used by AlphaGo’s creators at Google subsidiary DeepMind was to create an artificial intelligence program that could learn how to identify favourable moves from useless ones. This meant it wouldn’t have to analyse all the possible moves that could be made at each turn. In preparation for its first match against professional Go player Lee Sedol, AlphaGo analysed <a href="https://www.wired.com/2017/05/googles-alphago-levels-board-games-power-grids">around 300m moves</a> made by professional Go players. It then used what are called deep learning and reinforcement learning techniques to <a href="https://blog.google/topics/machine-learning/what-we-learned-in-seoul-with-alphago/">develop its own ability</a> to identify favourable moves.</p>
<p>But this wasn’t enough to enable AlphaGo to defeat highly ranked human players. The software was run on custom microchips specifically designed for machine learning, known as tensor processing units (TPUs), to support very large numbers of computations. This seems similar to the approach used by the designers of DeepBlue, who also developed custom chips for high-volume computation. The stark difference, however, is that DeepBlue’s chips could only be used for playing chess. AlphaGo’s chips run Google’s general-purpose AI framework, Tensorflow, and are also used to <a href="https://cloudplatform.googleblog.com/2016/05/Google-supercharges-machine-learning-tasks-with-custom-chip.html">power other Google services</a> such as Street View and optimisation tasks in the firm’s data centres.</p>
<h2>Lesson for us all</h2>
<p>The other thing that has changed since DeepBlue’s victory is the respect that humans have for their computer opponents. When playing chess computers, it was common for the human players to adopt so-called <a href="https://www.chess.com/blog/ramin18/anti-computer-tactics-gaming">anti-computer tactics</a>. This involves making conservative moves to prevent the computer from evaluating positions effectively.</p>
<p>In his first match against AlphaGo, however, Ke Jie, adopted tactics that had previously been used by his opponent to <a href="https://www.wired.com/2017/05/revamped-alphago-wins-first-game-chinese-go-grandmaster/">beat it at its own game</a>. Although this attempt failed, it demonstrates a change in approach for leading human players taking on computers. Instead of trying to stifle the machine, they have begun trying to learn from how it played in the past.</p>
<p>In fact, the machine has already influenced the professional game of Go, with grandmasters <a href="https://deepmind.com/blog/exploring-mysteries-alphago/">adopting AlphaGo’s strategy</a> during their tournament matches. This machine has taught humanity something new about a game it has been playing for over 2,500 years, liberating us from the experience of millennia.</p>
<p>What then might the future hold for the AI behind AlphaGo? The success of DeepBlue <a href="https://theconversation.com/twenty-years-on-from-deep-blue-vs-kasparov-how-a-chess-match-started-the-big-data-revolution-76882">triggered rapid developments</a> that have directly impacted the techniques applied in big data processing. The benefit of the technology used to implement AlphaGo is that it can already be applied to other problems that require pattern identification.</p>
<p>For example, the same techniques have been applied to <a href="https://www.wired.com/2017/05/using-ai-detect-cancer-not-just-cats/">the detection of cancer</a> and to create robots that can learn to do <a href="https://www.wired.com/2017/01/googles-go-playing-machine-opens-door-robots-learn/">things like open doors</a>, among <a href="https://www.wired.com/2017/01/googles-go-playing-machine-opens-door-robots-learn/">many other applications</a>. The underlying framework used in AlphaGo, Google’s TensorFlow, has been made freely available for developers and researchers to build new machine-learning programs using standard computer hardware. </p>
<p>More excitingly, combining it with the many computers available through the internet cloud creates the promise of delivering <a href="https://cloud.google.com/tpu/">machine-learning supercomputing</a>. When this technology matures then the potential will exist for the creation of self-taught machines in wide-ranging roles that can support complex decision-making tasks. Of course, what may be even more profound are the social impacts of having machines that not only teach themselves but teach us in the process.</p><img src="https://counter.theconversation.com/content/78410/count.gif" alt="The Conversation" width="1" height="1" />
<p class="fine-print"><em><span>Mark Robert Anderson does not work for, consult, own shares in or receive funding from any company or organisation that would benefit from this article, and has disclosed no relevant affiliations beyond their academic appointment.</span></em></p>Google’s AlphaGo victory over the human world champion shows how far things have come since DeepBlue.Mark Robert Anderson, Professor in Computing and Information Systems, Edge Hill UniversityLicensed as Creative Commons – attribution, no derivatives.tag:theconversation.com,2011:article/773832017-05-11T01:03:54Z2017-05-11T01:03:54ZComputers to humans: Shall we play a game?<figure><img src="https://images.theconversation.com/files/168795/original/file-20170510-21596-p2i8u6.png?ixlib=rb-1.1.0&q=45&auto=format&w=496&fit=clip" /><figcaption><span class="caption">Artificial intelligence can bring many benefits to human gamers.</span> <span class="attribution"><a class="source" href="https://www.instagram.com/gamingartbysj/">Sam Jordan Belanger</a>, <a class="license" href="http://creativecommons.org/licenses/by-nd/4.0/">CC BY-ND</a></span></figcaption></figure><p>Way back in the 1980s, a schoolteacher challenged me to write a computer program that played tic-tac-toe. I failed miserably. But just a couple of weeks ago, I explained to one of my computer science graduate students how to solve tic-tac-toe using the so-called “<a href="https://en.wikipedia.org/wiki/Minimax">Minimax algorithm</a>,” and it took us about an hour to write a program to do it. Certainly my coding skills have improved over the years, but computer science has come a long way too.</p>
<p>What seemed impossible just a couple of decades ago is startlingly easy today. In 1997, people were stunned when a chess-playing IBM computer named <a href="http://www.nytimes.com/1997/05/12/nyregion/swift-and-slashing-computer-topples-kasparov.html">Deep Blue beat international grandmaster Garry Kasparov</a> in a six-game match. In 2015, Google revealed that its DeepMind system had mastered several <a href="http://www.techrepublic.com/article/google-ai-beats-humans-at-more-classic-arcade-games-than-ever-before/">1980s-era video games</a>, including teaching itself a crucial winning strategy in “<a href="https://www.youtube.com/watch?v=V1eYniJ0Rnk">Breakout</a>.” In 2016, Google’s AlphaGo system beat a top-ranked Go player in a <a href="https://www.theatlantic.com/technology/archive/2016/03/the-invisible-opponent/475611/">five-game tournament</a>.</p>
<figure>
<iframe width="440" height="260" src="https://www.youtube.com/embed/V1eYniJ0Rnk?wmode=transparent&start=0" frameborder="0" allowfullscreen=""></iframe>
<figcaption><span class="caption">An artificial intelligence system learns to play ‘Breakout.’</span></figcaption>
</figure>
<p>The quest for technological systems that can beat humans at games continues. In late May, AlphaGo will take on <a href="https://arstechnica.com/information-technology/2017/04/deepmind-alphago-go-ke-jie-china/">Ke Jie</a>, the best player in the world, among other opponents at the Future of Go Summit in Wuzhen, China. With increasing computing power, and improved engineering, computers can beat humans even at games we thought relied on human intuition, wit, deception or bluffing – like <a href="http://www.csd.cs.cmu.edu/news/carnegie-mellon-ai-takes-chinese-poker-players">poker</a>. I recently saw a video in which volleyball players practice their serves and spikes against <a href="https://www.youtube.com/watch?v=EHKv6lRRV10">robot-controlled</a> rubber arms trying to block the shots. One lesson is clear: When machines play to win, human effort is futile. </p>
<figure>
<iframe width="440" height="260" src="https://www.youtube.com/embed/EHKv6lRRV10?wmode=transparent&start=0" frameborder="0" allowfullscreen=""></iframe>
<figcaption><span class="caption">Robots play volleyball.</span></figcaption>
</figure>
<p>This can be great: We want a perfect AI to drive our cars, and a tireless system looking for signs of cancer in X-rays. But when it comes to play, we don’t want to lose. Fortunately, AI can make games more fun, and perhaps even endlessly enjoyable.</p>
<h2>Designing games that never get old</h2>
<p>Today’s game designers – who write releases that <a href="http://www.businessinsider.com/here-are-the-top-10-highest-grossing-video-games-of-all-time-2012-6">earn more than a blockbuster movie</a> – see a problem: Creating an unbeatable artificial intelligence system is pointless. Nobody wants to play a game they have no chance of winning.</p>
<p>But people do want to play <a href="https://theconversation.com/the-future-is-in-interactive-storytelling-76772">games that are immersive, complex and surprising</a>. Even today’s best games become stale after a person plays for a while. The ideal game will engage players by adapting and reacting in ways that keep the game interesting, maybe forever.</p>
<p>So when we’re designing artificial intelligence systems, we should look not to the triumphant Deep Blues and AlphaGos of the world, but rather to the overwhelming success of massively multiplayer online games like “<a href="https://worldofwarcraft.com/en-us/">World of Warcraft</a>.” These sorts of games are graphically well-designed, but their key attraction is interaction. </p>
<p>It seems as if most people are not drawn to extremely difficult logical puzzles like chess and Go, but rather to meaningful connections and communities. The real challenge with these massively multi-player online games is not whether they can be beaten by intelligence (human or artificial), but rather how to keep the experience of playing them fresh and new every time.</p>
<h2>Change by design</h2>
<p>At present, game environments allow people lots of possible interactions with other players. The roles in a dungeon <a href="https://en.wikipedia.org/wiki/Raid_(gaming)">raiding party</a> are well-defined: Fighters take the damage, healers help them recover from their injuries and the fragile wizards cast spells from afar. Or think of “<a href="https://en.wikipedia.org/wiki/Portal_2">Portal 2</a>,” a game focused entirely on collaborating robots puzzling their way through a maze of cognitive tests.</p>
<p>Exploring these worlds together allows you to form common memories with your friends. But any changes to these environments or the underlying plots have to be made by human designers and developers.</p>
<p>In the real world, changes happen naturally, without supervision, design or manual intervention. Players learn, and living things adapt. Some organisms even <a href="http://dx.doi.org/10.1086/691101">co-evolve</a>, reacting to each other’s developments. (A similar phenomenon happens in a <a href="http://www.amnh.org/exhibitions/einstein/peace-and-war/nuclear-arms-race/">weapons technology arms race</a>.)</p>
<p>Computer games today lack that level of sophistication. And for that reason, I don’t believe developing an artificial intelligence that can play modern games will meaningfully advance AI research. </p>
<h2>We crave evolution</h2>
<p>A game worth playing is a game that is unpredictable because it adapts, a game that is ever novel because novelty is created by playing the game. Future games need to evolve. Their characters shouldn’t just react; they need to explore and learn to exploit weaknesses or cooperate and collaborate. <a href="http://www.livescience.com/474-controversy-evolution-works.html">Darwinian evolution and learning</a>, we understand, are the drivers of all novelty on Earth. It could be what <a href="https://theconversation.com/evolving-our-way-to-artificial-intelligence-54100">drives change in virtual environments</a> as well.</p>
<p>Evolution figured out how to create <a href="https://theconversation.com/understanding-the-four-types-of-ai-from-reactive-robots-to-self-aware-beings-67616">natural intelligence</a>. Shouldn’t we, instead of trying to code our way to AI, just evolve AI instead? Several labs – <a href="http://hintzelab.msu.edu/">including my own</a> and that of <a href="http://adamilab.msu.edu/">my colleague Christoph Adami</a> – are working on what is called “<a href="https://en.wikipedia.org/wiki/Neuroevolution">neuro-evolution</a>.”</p>
<p>In a computer, we simulate complex environments, like a road network or a biological ecosystem. We create virtual creatures and challenge them to evolve over hundreds of thousands of simulated generations. Evolution itself then develops the best drivers, or the best organisms at adapting to the conditions – those are the ones that survive. </p>
<figure>
<iframe width="440" height="260" src="https://www.youtube.com/embed/5lJuEW-5vr8?wmode=transparent&start=0" frameborder="0" allowfullscreen=""></iframe>
<figcaption><span class="caption">A neuro-evolution learns to drive a car.</span></figcaption>
</figure>
<p>Today’s AlphaGo is beginning this process, learning by continuously <a href="https://www.theguardian.com/technology/2016/jun/27/alphago-deepmind-ai-code-google">playing games against itself</a>, and by analyzing records of games played by top Go champions. But it does not learn while playing in the same way we do, experiencing unsupervised experimentation. And it doesn’t adapt to a particular opponent: For these computer players, the best move is the best move, regardless of an opponent’s style. </p>
<p>Programs that learn from experience are the next step in AI. They would make computer games much more interesting, and enable robots to not only function better in the real world, but to adapt to it on the fly.</p><img src="https://counter.theconversation.com/content/77383/count.gif" alt="The Conversation" width="1" height="1" />
<p class="fine-print"><em><span>Arend Hintze receives funding from NSF BEACON Center for the Study of Evolution in Action Cooperative Agreement No. DBI-0939454, and received funding from Strength in Numbers Game Studio </span></em></p>Twenty years after Deep Blue beat Garry Kasparov at chess, artificial intelligence can make games more fun, and perhaps even endlessly enjoyable, if it learns to adapt.Arend Hintze, Assistant Professor of Integrative Biology & Computer Science and Engineering, Michigan State UniversityLicensed as Creative Commons – attribution, no derivatives.tag:theconversation.com,2011:article/717132017-01-30T21:58:38Z2017-01-30T21:58:38ZKnow when to fold ‘em: AI beats world’s top poker players<figure><img src="https://images.theconversation.com/files/154676/original/image-20170130-8245-1g9gcqr.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=496&fit=clip" /><figcaption><span class="caption">Poker is a harder game for computers to master than chess or Go.</span> <span class="attribution"><span class="source">Shutterstock</span></span></figcaption></figure><p>If you were about to start playing a game of online poker, you might want to think again. Humankind has just been beaten at yet another game, this time <a href="http://wizardofodds.com/games/heads-up-hold-em/">Heads-Up No-Limit Texas Hold’em poker</a>. This is a milestone moment for artificial intelligence (<a href="https://theconversation.com/au/topics/artificial-intelligence-90">AI</a>). </p>
<p>The first game that humans lost to machines was backgammon. In 1979, the world backgammon champion was beaten by Hans Berliner’s <a href="http://www.bkgm.com/articles/Berliner/BackgammonProgramBeatsWorldChamp/">BKG 9.8</a> program.</p>
<p>In 1997, Gary Kasparov who was the reigning world chess champion <a href="http://content.time.com/time/subscriber/article/0,33009,984304,00.html">lost to IBM’s Deep Blue</a> program. Kasparov remarked that he could “smell” a new form of intelligence across the table from him.</p>
<p>Other games have since <a href="http://academicworks.cuny.edu/cgi/viewcontent.cgi?article=1181&context=gc_pubs">fallen to the machines</a>: <a href="http://www.thinkartificial.org/artificial-intelligence/the-unbeatable-checkers-ai-system/">Checkers</a>, <a href="http://academicworks.cuny.edu/cgi/viewcontent.cgi?article=1181&context=gc_pubs">Othello</a>, <a href="http://www.smp.uq.edu.au/sites/smp.uq.edu.au/files/WorldChampionshipScrabble.pdf">Scrabble</a>,the general knowledge quiz <a href="https://theconversation.com/have-computers-finally-eclipsed-their-creators-10">Jeopardy!</a>, even the classic arcade game <a href="http://thefutureofai.blogspot.de/2015/02/liefhacker-how-computers-taught.html">Pong</a>.</p>
<p>Most recently, the ancient Chinese board game of <a href="http://thefutureofai.blogspot.de/2016/03/the-conversation-ai-has-beaten-us-at-go.html">Go fell to the machines</a>. In March last year, one of the leading Go players on the planet, Lee Sedol, was beaten 4-1 by Google’s AlphaGo program.</p>
<p>And to rub our faces in it, over the Christmas break, AlphaGo anonymously played dozens of the world’s leading Go players online and <a href="http://thefutureofai.blogspot.de/2017/01/techrepublic-googles-ai-powered-alphago.html">won convincingly</a>.</p>
<h2>Why poker?</h2>
<p>Go has been described as the Mount Everest of board games. It is far more complex than chess or many other games. However, it is less of a challenge than poker.</p>
<p>Like the real world, poker is a game of uncertainty. Players don’t know what cards the other players have. Or what cards will be dealt in the future. In a game like chess or Go, by comparison, all the players can see the board. Everyone has complete information. This makes Chess and Go much easier to program than poker.</p>
<p>Poker also requires understanding the psychology of the other players. Are they bluffing? Should you fold? Should you bluff?</p>
<p>Finally poker involves betting. When should you bet? What should you bet? This again adds to the challenge of writing a poker program that plays as well as or better than humans.</p>
<p>Over the last three weeks, four of the top poker players have been locked in an exhausting <a href="https://www.riverscasino.com/pittsburgh/BrainsVsAI/">120,000 game match</a> at the Rivers Casino in Pittsburgh.</p>
<p>Their opponent is Carnegie Mellon University’s <a href="https://www.cmu.edu/news/stories/archives/2017/january/AI-tough-poker-player.html">Libratus program</a>, written by my colleague <a href="http://www.cs.cmu.edu/%7Esandholm/">Professor Tuomas Sandholm</a> and his PhD student <a href="http://www.cs.cmu.edu/%7Enoamb/">Noam Brown</a>.</p>
<figure>
<iframe width="440" height="260" src="https://www.youtube.com/embed/JtyA2aUj4WI?wmode=transparent&start=0" frameborder="0" allowfullscreen=""></iframe>
</figure>
<p>Libratus is set to win the tournament later today, finishing ahead of the humans with more than US$1 million (A$1.32m) of notional winnings. The pros can be consoled by sharing out the actual US$200,000 (A$265,000) prize pot.</p>
<p>In order to reduce the influence of sheer luck on the result, the tournament used <a href="http://www.computerpokercompetition.org/index.php/competitions/results/81-background/76-duplicate-poker">duplicate hands</a>. This means that two decks of identically shuffled cards are used to two separate tables. On one table, a human player is simultaneously dealt their hand, call it hand A, and the AI is dealt hand B. On the other table (situated in another room), the AI player is dealt hand A and human player dealt hand B.</p>
<p>This means that even if one player receives an unusual number of lucky hands, then this will be mirrored for the other player in the duplicate game. </p>
<p>This also explains why so many games have been played. The end result is that we can say with statistical confidence that Libratus is better than the human players.</p>
<h2>How to win at poker</h2>
<p>The details of how Libratus plays are still secret. But we can make some educated guesses based on the <a href="http://www.cs.cmu.edu/%7Eamem/">Carnegie Mellon University team’s</a> previous work.</p>
<p>Perhaps most interesting is that the victory depends more on Good Old Fashioned AI (GOFAI) than on <a href="http://thefutureofai.blogspot.de/2016/06/the-conversation-business-is-waking-up.html">the currently fashionable deep learning</a> processes.</p>
<p>Like IBM’s Deep Blue in chess, Libratus used a lot of brute force calculation as to how to play best. We know it calls upon Pittsburgh’s Supercomputing Centre to play out every end game.</p>
<p>And each night, Libratus uses this supercomputer to refine its strategy. In case you think this is unfair on the humans, the pros also get together at night after each match to compare performance and plan for the next day.</p>
<p>Libratus also takes advantage of game theory, the branch of mathematics made famous by the movie <a href="http://www.imdb.com/title/tt0268978/">A Beautiful Mind</a> about John Nash. Libratus looks to play strategic moves that cannot be bettered whatever its opponent does.</p>
<h2>What next?</h2>
<p>Poker is still not solved. Libratus only plays the two player version of Heads-Up No-Limit Texas Hold’em poker. Adding more players increases the complexity greatly. So it will be a few years yet before computers can play well against four or more players.</p>
<p>But this is another example of how in narrow focused domains AI is starting to take over from humans: <a href="http://www.sciencealert.com/ai-analyses-mammograms-30-times-faster-and-20-more-accurately-than-doctors">reading mammograms</a>, <a href="https://www.technologyreview.com/s/544651/baidus-deep-learning-system-rivals-people-at-speech-recognition/">transcribing Chinese</a>, <a href="https://www.wired.com/2016/06/ai-fighter-pilot-beats-human-no-need-panic-really/">beating human pilots in dogfights</a>… the list increases almost weekly. </p>
<p>Not surprisingly, many people are wondering where this all ends. Will computers eventually <a href="https://theconversation.com/job-survival-in-the-age-of-robots-and-intelligent-machines-33906">take over all the jobs</a>?</p>
<p><a href="https://theconversation.com/dont-be-alarmed-ai-wont-leave-half-the-world-unemployed-54958">A widely reported study from the University of Oxford</a> in 2013 estimated that 47% of jobs in the US were at risk from automation in the next two decades.</p>
<p>There were several limitations in the Oxford study. Ironically, one was that it automated the task of predicting which jobs were under risk. The study used machine learning and a small training set of 70 hand labelled jobs to predict which of over 700 professions was under risk.</p>
<p>This is where you can help. I am calling on the wisdom of the crowd to see if we can make a better prediction. Please take a few minutes to <a href="https://goo.gl/forms/lTQryqWxgTp6rROr2">complete our survey</a>. At the end, you can nominate a charity to receive a donation in recognition of your time and effort.</p>
<p>Even before the results of our survey are in, its clear that some jobs such as taxi driver, truck driver, radiographer and now poker pro are under threat. Of course, technology will also create other new jobs. But whether as many get created or destroyed remains an interesting open question.</p>
<p>To keep ahead of the bots, humans will need to play to their strengths like creativity and emotional intelligence. We also should look to augment rather than replace humans. Together humans and machines can outperform machines or humans alone. The best chess player today is a human working with a computer.</p>
<p>Together, we can be super-human.</p><img src="https://counter.theconversation.com/content/71713/count.gif" alt="The Conversation" width="1" height="1" />
<p class="fine-print"><em><span>Toby Walsh does not work for, consult, own shares in or receive funding from any company or organisation that would benefit from this article, and has disclosed no relevant affiliations beyond their academic appointment.</span></em></p>Artificial intelligence researchers have upped the ante and developed a program that has beaten the world’s best Heads-Up No-Limit Texas Hold’em poker players.Toby Walsh, Professor of AI at UNSW, Research Group Leader, Data61Licensed as Creative Commons – attribution, no derivatives.tag:theconversation.com,2011:article/676162016-11-14T01:40:10Z2016-11-14T01:40:10ZUnderstanding the four types of AI, from reactive robots to self-aware beings<figure><img src="https://images.theconversation.com/files/143746/original/image-20161028-15775-i00zp.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=496&fit=clip" /><figcaption><span class="caption">Robots will need to teach themselves.</span> <span class="attribution"><a class="source" href="http://www.shutterstock.com/pic-350368274/">Robot reading via shutterstock.com</a></span></figcaption></figure><p>The common, and recurring, view of the latest breakthroughs in artificial intelligence research is that sentient and intelligent machines are just on the horizon. Machines understand verbal commands, distinguish pictures, drive cars and play games better than we do. How much longer can it be before they walk among us?</p>
<p>The new <a href="https://www.whitehouse.gov/sites/default/files/whitehouse_files/microsites/ostp/NSTC/preparing_for_the_future_of_ai.pdf">White House report on artificial intelligence</a> takes an appropriately skeptical view of that dream. It says the next 20 years likely won’t see machines “exhibit broadly-applicable intelligence comparable to or exceeding that of humans,” though it does go on to say that in the coming years, “machines will reach and exceed human performance on more and more tasks.” But its assumptions about how those capabilities will develop missed some important points.</p>
<p>As an AI researcher, I’ll admit it was nice to have my own field highlighted at the highest level of American government, but the report focused almost exclusively on what I call “the boring kind of AI.” It dismissed in half a sentence my branch of AI research, into how evolution can help develop ever-improving AI systems, and how computational models can help us understand how our human intelligence evolved.</p>
<p>The report focuses on what might be called mainstream AI tools: machine learning and deep learning. These are the sorts of technologies that have been able to <a href="http://dx.doi.org/10.1016/S0004-3702(01)00129-1">play “Jeopardy!” well</a>, and <a href="http://dx.doi.org/10.1038/nature16961">beat human Go masters</a> at the most complicated game ever invented. These current intelligent systems are able to handle huge amounts of data and make complex calculations very quickly. But they lack an element that will be key to building the sentient machines we picture having in the future.</p>
<p>We need to do more than teach machines to learn. We need to overcome the boundaries that define the four different types of artificial intelligence, the barriers that separate machines from us – and us from them.</p>
<h2>Type I AI: Reactive machines</h2>
<p>The most basic types of AI systems are purely reactive, and have the ability neither to form memories nor to use past experiences to inform current decisions. <a href="http://www.techrepublic.com/article/ibm-watson-the-inside-story-of-how-the-jeopardy-winning-supercomputer-was-born-and-what-it-wants-to-do-next/">Deep Blue, IBM’s chess-playing supercomputer</a>, which beat international grandmaster Garry Kasparov in the late 1990s, is the perfect example of this type of machine. </p>
<p>Deep Blue can identify the pieces on a chess board and know how each moves. It can make predictions about what moves might be next for it and its opponent. And it can choose the most optimal moves from among the possibilities.</p>
<p>But it doesn’t have any concept of the past, nor any memory of what has happened before. Apart from a rarely used chess-specific rule against repeating the same move three times, Deep Blue ignores everything before the present moment. All it does is look at the pieces on the chess board as it stands right now, and choose from possible next moves.</p>
<p>This type of intelligence involves the computer <a href="https://www.youtube.com/watch?v=t3kXWSctj2Q">perceiving the world directly</a> and acting on what it sees. It doesn’t rely on an internal concept of the world. In a seminal paper, AI researcher Rodney Brooks argued that <a href="http://dx.doi.org/10.1016/0004-3702(91)90053-M">we should only build machines</a> like this. His main reason was that people are not very good at programming accurate simulated worlds for computers to use, what is called in AI scholarship a “representation” of the world.</p>
<p>The current intelligent machines we marvel at either have no such concept of the world, or have a very limited and specialized one for its particular duties. The <a href="https://www.scientificamerican.com/article/how-the-computer-beat-the-go-master/">innovation in Deep Blue’s design</a> was not to broaden the range of possible movies the computer considered. Rather, the developers found a way to narrow its view, to <a href="https://www.cnet.com/news/did-a-bug-in-deep-blue-lead-to-kasparovs-defeat/">stop pursuing some potential future moves</a>, based on how it rated their outcome. Without this ability, Deep Blue would have needed to be an even more powerful computer to actually beat Kasparov.</p>
<p>Similarly, Google’s AlphaGo, which has beaten top human Go experts, can’t evaluate all potential future moves either. Its analysis method is more sophisticated than Deep Blue’s, using a <a href="http://pages.cs.wisc.edu/%7Ebolo/shipyard/neural/local.html">neural network</a> to evaluate game developments. </p>
<p>These methods do improve the ability of AI systems to play specific games better, but they can’t be easily changed or applied to other situations. These computerized imaginations have no concept of the wider world – meaning they can’t function beyond the specific tasks they’re assigned and are <a href="http://dx.doi.org/10.1109/CVPR.2015.7298640">easily fooled</a>. </p>
<p>They can’t interactively participate in the world, the way we imagine AI systems one day might. Instead, these machines will behave exactly the same way every time they encounter the same situation. This can be very good for ensuring an AI system is trustworthy: You want your autonomous car to be a reliable driver. But it’s bad if we want machines to truly engage with, and respond to, the world. These simplest AI systems won’t ever be bored, or interested, or sad.</p>
<h2>Type II AI: Limited memory</h2>
<p>This Type II class contains machines can look into the past. Self-driving cars do some of this already. For example, they observe other cars’ speed and direction. That can’t be done in a just one moment, but rather requires identifying specific objects and monitoring them over time.</p>
<p>These observations are added to the self-driving cars’ preprogrammed representations of the world, which also include lane markings, traffic lights and other important elements, like curves in the road. They’re included when the car decides when to change lanes, to avoid cutting off another driver or being hit by a nearby car. </p>
<p>But these simple pieces of information about the past are only transient. They aren’t saved as part of the car’s library of experience it can learn from, the way human drivers compile experience over years behind the wheel.</p>
<p>So how can we build AI systems that build full representations, remember their experiences and learn how to handle new situations? Brooks was right in that it is very difficult to do this. My own research into methods inspired by Darwinian evolution can start to <a href="http://dx.doi.org/10.1162/NECO_a_00475">make up for human shortcomings</a> by letting the machines build their own representations.</p>
<h2>Type III AI: Theory of mind</h2>
<p>We might stop here, and call this point the important divide between the machines we have and the machines we will build in the future. However, it is better to be more specific to discuss the types of representations machines need to form, and what they need to be about.</p>
<p>Machines in the next, more advanced, class not only form representations about the world, but also about other agents or entities in the world. In psychology, this is called “<a href="http://dx.doi.org/10.1017/S0140525X00076512">theory of mind</a>” – the understanding that people, creatures and objects in the world can have thoughts and emotions that affect their own behavior.</p>
<p>This is crucial to <a href="https://theconversation.com/can-great-apes-read-your-mind-66224">how we humans formed societies</a>, because they allowed us to have social interactions. Without understanding each other’s motives and intentions, and without taking into account what somebody else knows either about me or the environment, working together is at best difficult, at worst impossible. </p>
<p>If AI systems are indeed ever to walk among us, they’ll have to be able to understand that each of us has thoughts and feelings and expectations for how we’ll be treated. And they’ll have to adjust their behavior accordingly.</p>
<h2>Type IV AI: Self-awareness</h2>
<p>The final step of AI development is to build systems that can form representations about themselves. Ultimately, we AI researchers will have to not only understand consciousness, but build machines that have it. </p>
<p>This is, in a sense, an extension of the “theory of mind” possessed by Type III artificial intelligences. Consciousness is also called “self-awareness” for a reason. (“I want that item” is a very different statement from “I know I want that item.”) Conscious beings are aware of themselves, know about their internal states, and are able to predict feelings of others. We assume someone honking behind us in traffic is angry or impatient, because that’s how we feel when we honk at others. Without a theory of mind, we could not make those sorts of inferences.</p>
<p>While we are probably far from creating machines that are self-aware, we should focus our efforts toward understanding memory, learning and the ability to base decisions on past experiences. This is an important step to understand human intelligence on its own. And it is crucial if we want to design or evolve machines that are more than exceptional at classifying what they see in front of them.</p><img src="https://counter.theconversation.com/content/67616/count.gif" alt="The Conversation" width="1" height="1" />
<p class="fine-print"><em><span>Arend Hintze works for Michigan State University. He receives funding from NSF and Strength in Numbers Game Company to research AI. </span></em></p>We need to do more than teach machines to learn. We need to overcome the barriers that separate machines from us – and us from them.Arend Hintze, Assistant Professor of Integrative Biology & Computer Science and Engineering, Michigan State UniversityLicensed as Creative Commons – attribution, no derivatives.tag:theconversation.com,2011:article/622962016-07-13T13:03:25Z2016-07-13T13:03:25ZWhy football, not chess, is the true final frontier for robotic artificial intelligence<p>The perception of what artificial intelligence was capable of began to change when chess grand master and world champion <a href="http://www-03.ibm.com/ibm/history/ibm100/us/en/icons/deepblue/">Garry Kasparov lost to Deep Blue</a>, IBM’s chess-playing program, in 1997. Deep Blue, it was felt, had breached the domain of a cerebral activity considered the exclusive realm of human intellect. This was not because of something technologically new: in the end, chess was felled by the brute force of faster computers and clever heuristics. But if chess is considered the game of kings, then the east Asian board game Go is the game of emperors. </p>
<p>Significantly more complex, requiring even more strategic thinking, and featuring an intricate interweaving of tactical and strategical components, it posed an even greater challenge to artificial intelligence. Go relies much more on pattern recognition and subtle evaluation of the general positions of playing pieces. With a number of possible moves per turn an order of magnitude greater than chess, any algorithm trying to evaluate all possible future moves was expected to fail. </p>
<p>Until the early 2000s, programs playing Go progressed slowly, and could be beaten by amateurs. But this changed in 2006, with the introduction of two new techniques. First was the <a href="https://jeffbradberry.com/posts/2015/09/intro-to-monte-carlo-tree-search/">Monte Carlo tree search</a>, an algorithm that rather than attempting to examine all possible future moves instead tests a sparse selection of them, combining their value in a sophisticated way to get a better estimate of a move’s quality. The second was the (re)discovery of deep networks, a contemporary incarnation of neural networks that had been experimented with since the 1960s, but which was now cheaper, more powerful, and equipped with huge amounts of data with which to train the learning algorithms.</p>
<p>The combination of these techniques saw a drastic improvement in Go-playing programs, and ultimately <a href="https://theconversation.com/googles-go-triumph-is-a-milestone-for-artificial-intelligence-research-53762">Google DeepMind’s AlphaGo program beat Go world champion Lee Sedol</a> in March 2016. Now that Go has fallen, where do we go from here?</p>
<h2>The future of AI is in physical form</h2>
<p>Following Kasparov’s defeat in 1997, scientists considered that the challenge for AI was not to conquer some cerebral game. Rather, it needed to be physically embodied in the real world: football.</p>
<p>Football is easy for humans to pick up, but to have a humanoid robot running around a field on two legs, seeing and taking control of the ball, communicating under pressure with teammates, and all mostly without falling over, was considered completely out of the question in 1997. Only a handful of laboratories were able to design a walking humanoid robot. Led by <a href="http://sbiaustralia.org/systems-biology/kitanoprofile/">Hiroaki Kitano</a> and <a href="https://www.cmu.edu/me/people/veloso.html">Manuela Veloso</a>, the ambitious goal set that year was to have by 2050 a team of humanoid robots able to play a game of football against the world champion team according to FIFA rules, and win. And so the RoboCup competition was born.</p>
<figure>
<iframe width="440" height="260" src="https://www.youtube.com/embed/tAd1IeovyY8?wmode=transparent&start=0" frameborder="0" allowfullscreen=""></iframe>
</figure>
<p>The <a href="http://www.robocup.org/">RoboCup tournament</a> held its <a href="http://www.robocup2016.org/en/">20th competition in Leipzig this year</a>. Its goal has always been to improve and challenge the capacity of artificial intelligence and robotics, not in the abstract but in the much more challenging form of physical robots that act and interact with others in real time. In the years since, many other organisations have <a href="https://theconversation.com/cybathlon-will-showcase-what-bionics-could-do-for-millions-with-disabilities-54760">recognised how such competitions boost technological progress</a>.</p>
<p>The first RoboCup featured only wheeled robots and simulated 2D football leagues, but soon leagues that permitted Sony’s <a href="http://www.sony-aibo.com/">four-legged AIBO robot dogs</a> were introduced and, since 2003, <a href="http://wiki.robocup.org/wiki/Humanoid_League">humanoid leagues</a>. In the beginning, the humanoids’ game was quite limited, with very shaky robots attempting quivering steps, and where kicking the ball almost invariably caused the robot to fall. In recent years, their ability has significantly improved: many labs now boast <a href="http://robocup.herts.ac.uk/">five or six-a-side humanoid robot teams</a>.</p>
<figure>
<iframe width="440" height="260" src="https://www.youtube.com/embed/pzYHAp7b7sY?wmode=transparent&start=0" frameborder="0" allowfullscreen=""></iframe>
</figure>
<h2>No ordinary ballgame</h2>
<p>In order to push competitors on to reach the goal of a real football match by 2050, the conditions are made harder every year. Last year, the green carpet was replaced by artificial turf, and the goalposts and the ball coloured white. This makes it harder for robots to maintain stability and poses a challenge of recognising the goals and ball. So while the robots may seem less capable this year than the year before, it’s because the goalposts are moving.</p>
<p>The tasks involved in playing football, although much more intuitive to humans than chess or Go, are a major challenge for robots. Technical problems of hitherto unimaginable complexity have to be solved: timing a kick while running, identifying the ball against a glaring sun, running on wet grass, providing the robot with sufficient energy for 45 minutes’ play, even the materials that go into constructing a robot can’t disintegrate during a forceful game. Other problems to be solved will define important aspects of our life with robots in the future: when a robot collides with a human player, who can take how much damage? If humans commit fouls, may a robot foul back? </p>
<p>RoboCup offers up in miniature the problems we face as we head towards intelligent robots interacting with humans. It is not in the cerebral boardgames of chess or Go, but here on the pitch in the physical game of football that the frontline of life with intelligent robots is being carved out.</p><img src="https://counter.theconversation.com/content/62296/count.gif" alt="The Conversation" width="1" height="1" />
<p class="fine-print"><em><span>Daniel Polani has been heading teams participating at the RoboCup competition since 1998. He was member of the executive, later the trustee board and is now president elect of the RoboCup Federation.</span></em></p>Computers must master football if they are to demonstrate that they can be our equal.Daniel Polani, Professor of Artificial Intelligence, University of HertfordshireLicensed as Creative Commons – attribution, no derivatives.tag:theconversation.com,2011:article/605552016-07-05T20:03:43Z2016-07-05T20:03:43ZIf machines can beat us at games, does it make them more intelligent than us?<figure><img src="https://images.theconversation.com/files/128800/original/image-20160630-15259-1c34fkw.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=496&fit=clip" /><figcaption><span class="caption">Computers are getting better at playing games such as chess.</span> <span class="attribution"><span class="source">Shutterstock/Vasilyev Alexandr</span></span></figcaption></figure><p>The year 1997 saw the ultimate man versus machine tournament, with chess grandmaster Garry Kasparov losing to a machine called <a href="http://www-03.ibm.com/ibm/history/ibm100/us/en/icons/deepblue/">Deep Blue</a>. </p>
<p>Earlier this year, in what was hailed as another breakthrough in artificial intelligence (AI) research, Google’s <a href="https://deepmind.com/alpha-go">AlphaGo</a> defeated a professional Go player. </p>
<p>Go is an ancient Chinese board game that has hitherto been difficult for a computer to play at a high level due to its deceptively complex gameplay. Where chess is played on a board of 8 x 8 squares, Go is typically played on a board of 19 x 19 squares. </p>
<figure class="align-center zoomable">
<a href="https://images.theconversation.com/files/126215/original/image-20160611-29203-et35mh.png?ixlib=rb-1.1.0&q=45&auto=format&w=1000&fit=clip"><img alt="" src="https://images.theconversation.com/files/126215/original/image-20160611-29203-et35mh.png?ixlib=rb-1.1.0&q=45&auto=format&w=754&fit=clip" srcset="https://images.theconversation.com/files/126215/original/image-20160611-29203-et35mh.png?ixlib=rb-1.1.0&q=45&auto=format&w=600&h=600&fit=crop&dpr=1 600w, https://images.theconversation.com/files/126215/original/image-20160611-29203-et35mh.png?ixlib=rb-1.1.0&q=30&auto=format&w=600&h=600&fit=crop&dpr=2 1200w, https://images.theconversation.com/files/126215/original/image-20160611-29203-et35mh.png?ixlib=rb-1.1.0&q=15&auto=format&w=600&h=600&fit=crop&dpr=3 1800w, https://images.theconversation.com/files/126215/original/image-20160611-29203-et35mh.png?ixlib=rb-1.1.0&q=45&auto=format&w=754&h=754&fit=crop&dpr=1 754w, https://images.theconversation.com/files/126215/original/image-20160611-29203-et35mh.png?ixlib=rb-1.1.0&q=30&auto=format&w=754&h=754&fit=crop&dpr=2 1508w, https://images.theconversation.com/files/126215/original/image-20160611-29203-et35mh.png?ixlib=rb-1.1.0&q=15&auto=format&w=754&h=754&fit=crop&dpr=3 2262w" sizes="(min-width: 1466px) 754px, (max-width: 599px) 100vw, (min-width: 600px) 600px, 237px"></a>
<figcaption>
<span class="caption">The 19 x 19 Go game board.</span>
<span class="attribution"><span class="source">Shutterstock/Peter Hermes Furian</span></span>
</figcaption>
</figure>
<p>These are all worthy engineering achievements, but what does it mean for research into genuine machine intelligence and the predicted artificial intelligence that will surpass human intelligence?</p>
<p>Arguably, not much. To understand why, we need to delve a little more into the complexity of the games and the differences between how machines and humans play. </p>
<p>It is estimated the number of possible games of chess is 10<sup>120</sup> while the lowest limit of games for Go is 10<sup>360</sup>. These numbers are big, even for a computer. If you’re not quite convinced of this, the estimated number of atoms in the observable universe is merely 10<sup>79</sup> – minuscule in comparison. </p>
<p>Game-playing AI still cannot foresee every possible game play and, just like us, has to consider the options and make a decision on what move to make. For brevity, we’ll mainly stick with chess as it’s more widely known. Let’s look at how a computer plays first. </p>
<h2>The machine</h2>
<p>Most chess programs operate via brute-force search, which means they look through as many future positions as they can before before making a choice. </p>
<p>This results in a tree of possible combinations called the search tree. Here’s an example: </p>
<figure class="align-center zoomable">
<a href="https://images.theconversation.com/files/125336/original/image-20160606-25976-t46i9w.png?ixlib=rb-1.1.0&q=45&auto=format&w=1000&fit=clip"><img alt="" src="https://images.theconversation.com/files/125336/original/image-20160606-25976-t46i9w.png?ixlib=rb-1.1.0&q=45&auto=format&w=754&fit=clip" srcset="https://images.theconversation.com/files/125336/original/image-20160606-25976-t46i9w.png?ixlib=rb-1.1.0&q=45&auto=format&w=600&h=221&fit=crop&dpr=1 600w, https://images.theconversation.com/files/125336/original/image-20160606-25976-t46i9w.png?ixlib=rb-1.1.0&q=30&auto=format&w=600&h=221&fit=crop&dpr=2 1200w, https://images.theconversation.com/files/125336/original/image-20160606-25976-t46i9w.png?ixlib=rb-1.1.0&q=15&auto=format&w=600&h=221&fit=crop&dpr=3 1800w, https://images.theconversation.com/files/125336/original/image-20160606-25976-t46i9w.png?ixlib=rb-1.1.0&q=45&auto=format&w=754&h=277&fit=crop&dpr=1 754w, https://images.theconversation.com/files/125336/original/image-20160606-25976-t46i9w.png?ixlib=rb-1.1.0&q=30&auto=format&w=754&h=277&fit=crop&dpr=2 1508w, https://images.theconversation.com/files/125336/original/image-20160606-25976-t46i9w.png?ixlib=rb-1.1.0&q=15&auto=format&w=754&h=277&fit=crop&dpr=3 2262w" sizes="(min-width: 1466px) 754px, (max-width: 599px) 100vw, (min-width: 600px) 600px, 237px"></a>
<figcaption>
<span class="caption">An example of a search tree for a particular game.</span>
<span class="attribution"><span class="source">David Ireland</span>, <span class="license">Author provided</span></span>
</figcaption>
</figure>
<p>The search tree starts with a root that represents current game play. And the branches are all the possible game plays. Each level of the tree is called a ply, which is a single move by a player.</p>
<p>Deep Blue’s specialised hardware allowed it to search future game play at a staggering 200 million chess positions per second. Even today, most chess AI programs only compute about 5 million positions per second.</p>
<p>Not only does the AI have to search through a large collection of chess positions, but at some stage, it must evaluate them for their potential worth. This is done by a so-called evaluation function. </p>
<p>Deep Blue’s evaluation function was developed by a team of programmers and chess grandmasters who distilled their knowledge of chess into a function that evaluates piece strength, king safety, control of the centre, piece mobility, pawn structure and many other characteristics — everything a novice is taught.</p>
<p>This allows a particular board position to be scored with a single number. Think of the evaluation function as something like this:</p>
<figure class="align-center zoomable">
<a href="https://images.theconversation.com/files/125302/original/image-20160606-25980-xilyrs.png?ixlib=rb-1.1.0&q=45&auto=format&w=1000&fit=clip"><img alt="" src="https://images.theconversation.com/files/125302/original/image-20160606-25980-xilyrs.png?ixlib=rb-1.1.0&q=45&auto=format&w=754&fit=clip" srcset="https://images.theconversation.com/files/125302/original/image-20160606-25980-xilyrs.png?ixlib=rb-1.1.0&q=45&auto=format&w=600&h=159&fit=crop&dpr=1 600w, https://images.theconversation.com/files/125302/original/image-20160606-25980-xilyrs.png?ixlib=rb-1.1.0&q=30&auto=format&w=600&h=159&fit=crop&dpr=2 1200w, https://images.theconversation.com/files/125302/original/image-20160606-25980-xilyrs.png?ixlib=rb-1.1.0&q=15&auto=format&w=600&h=159&fit=crop&dpr=3 1800w, https://images.theconversation.com/files/125302/original/image-20160606-25980-xilyrs.png?ixlib=rb-1.1.0&q=45&auto=format&w=754&h=200&fit=crop&dpr=1 754w, https://images.theconversation.com/files/125302/original/image-20160606-25980-xilyrs.png?ixlib=rb-1.1.0&q=30&auto=format&w=754&h=200&fit=crop&dpr=2 1508w, https://images.theconversation.com/files/125302/original/image-20160606-25980-xilyrs.png?ixlib=rb-1.1.0&q=15&auto=format&w=754&h=200&fit=crop&dpr=3 2262w" sizes="(min-width: 1466px) 754px, (max-width: 599px) 100vw, (min-width: 600px) 600px, 237px"></a>
<figcaption>
<span class="caption">How a chess-playing AI evaluates the chessboard.</span>
<span class="attribution"><span class="source">David Ireland</span>, <span class="license">Author provided</span></span>
</figcaption>
</figure>
<p>The higher the number, the better the position is for the machine. The machine seeks to maximise this function in its favour, and minimise it for its opponent. </p>
<h2>The human</h2>
<p>A person, in stark contrast, only considers three to five positions per second, at best. How, then, did Kasparov give Deep Blue a run for its money? </p>
<p>This question has fascinated cognitive scientists who have yet to agree on a computational theory on how even an amateur plays chess. </p>
<p>Nevertheless, there’s been extensive psychological research into the cognitive processes involved in how players of various strengths perceive the chessboard and how they go about selecting a move.</p>
<p>Studies conducted on <a href="digitalcollections.library.cmu.edu/awweb/awarchive?type=file&item=44582">eye movements of expert players</a> as they select a move showed little consistency with searching a tree of possible moves. People, it seems, pay more attention to squares that contain active attacking and defending pieces and perceive the pieces on the board as groups or chunks rather than as individual pieces. </p>
<p>In an even more revealing experiment, <a href="http://www.ncbi.nlm.nih.gov/pmc/articles/PMC4142462/">novice and expert players</a> were shown a chess position taken from a game for five seconds. They were then asked to reproduce the board from memory. Expert players were able to reconstruct the board much more accurately than novice players. </p>
<p>Curiously, when they were asked to reconstruct a board that had the pieces randomly distributed, experts did no better than novices. </p>
<p>It is believed that through constant play, a player accumulates a large number of chunks that could be thought of as a language of chess. These chunks were not present with the randomly distributed board and, as such, the experts’ perception was no better than that of the novice.</p>
<p>This language encodes positions, threats, blocks, defences, attacks, forks and the many other complex combinations that arise. It allows players to determine and prioritise pressures on the board and reveal opportunities and dangers. </p>
<p>The language of chess is a higher level of perception of the chessboard that still eludes AI and cognitive science researchers.</p>
<p>Let’s take a look at an interesting position. </p>
<h2>What is white’s winning strategy?</h2>
<p>Two kings are on either side of a pawn blockade. White has an opportunity to promote the pawn on F6 to a stronger piece. But that square is being guarded by the black king.</p>
<figure class="align-center zoomable">
<a href="https://images.theconversation.com/files/125301/original/image-20160606-25996-1nat9te.png?ixlib=rb-1.1.0&q=45&auto=format&w=1000&fit=clip"><img alt="" src="https://images.theconversation.com/files/125301/original/image-20160606-25996-1nat9te.png?ixlib=rb-1.1.0&q=45&auto=format&w=754&fit=clip" srcset="https://images.theconversation.com/files/125301/original/image-20160606-25996-1nat9te.png?ixlib=rb-1.1.0&q=45&auto=format&w=600&h=593&fit=crop&dpr=1 600w, https://images.theconversation.com/files/125301/original/image-20160606-25996-1nat9te.png?ixlib=rb-1.1.0&q=30&auto=format&w=600&h=593&fit=crop&dpr=2 1200w, https://images.theconversation.com/files/125301/original/image-20160606-25996-1nat9te.png?ixlib=rb-1.1.0&q=15&auto=format&w=600&h=593&fit=crop&dpr=3 1800w, https://images.theconversation.com/files/125301/original/image-20160606-25996-1nat9te.png?ixlib=rb-1.1.0&q=45&auto=format&w=754&h=745&fit=crop&dpr=1 754w, https://images.theconversation.com/files/125301/original/image-20160606-25996-1nat9te.png?ixlib=rb-1.1.0&q=30&auto=format&w=754&h=745&fit=crop&dpr=2 1508w, https://images.theconversation.com/files/125301/original/image-20160606-25996-1nat9te.png?ixlib=rb-1.1.0&q=15&auto=format&w=754&h=745&fit=crop&dpr=3 2262w" sizes="(min-width: 1466px) 754px, (max-width: 599px) 100vw, (min-width: 600px) 600px, 237px"></a>
<figcaption>
<span class="caption">An example problem where human intuition often triumphs over AI.</span>
<span class="attribution"><span class="source">David Ireland</span>, <span class="license">Author provided</span></span>
</figcaption>
</figure>
<p>For white to win, the white king must move around the blockade via column A and force the black king away. Defeat for black is then inevitable. </p>
<p>Simple enough? Not entirely for a chess AI, which has more difficulty perceiving white’s advantage. This is because it would need to search to a depth of 20 ply to find white’s advantage. In this position, at 15 plies there are 10,142,484,904,590 possible positions (we tried computing to 20 plies but after one week of computation, we gave up).</p>
<p>Most computer chess programs won’t see the winning strategy. Instead, they will move the white king to the centre of the board which is the common strategy when there are only a few pieces on the board.</p>
<p>Human intuition is still a powerful force. </p>
<h2>Higher level of perception</h2>
<p>A famous AI researcher, <a href="http://www.soic.indiana.edu/all-people/profile.html?profile_id=229">Professor Douglas Hofstader</a>, believes analogy is the <a href="https://prelectur.stanford.edu/lecturers/hofstadter/analogy.html">core of cognition</a>. We humans, certainly bring our own analogies to the game: gambits, sacrifices, and blockades, among other things.</p>
<p>Alas, research into the field of cognitive science has waned over the past decade in favour of more practical and profitable direct AI approaches as seen in <a href="http://www.ibm.com/smarterplanet/us/en/ibmwatson/">Watson</a> and AlphaGo. </p>
<p>Nevertheless, there has been sporadic research output on so-called cognitive architectures (<a href="http://www.chrest.info/">CHREST</a>) that model human perception, learning, memory, and problem solving. </p>
<p>Some play chess <a href="https://chessprogramming.wikispaces.com/Chump">(CHUMP)</a> not by searching for a plethora of combinations but by perceiving patterns and relationships between pieces and squares. And just like most humans, they play mediocre chess. </p>
<p>It’s worth pondering: if true artificial intelligence is established, will it begin with an explosion of intelligence or something smaller and imperceptible?</p><img src="https://counter.theconversation.com/content/60555/count.gif" alt="The Conversation" width="1" height="1" />
<p class="fine-print"><em><span>David Ireland receives funding from Australian Research Council Centre of Excellence for the Dynamics of Language.</span></em></p>Artificial intelligence gives us machines that can beat humans at games such as chess and go. How long before we see AI surpass human intelligence?David Ireland, Electronic Engineer and Research Scientist at the Australian E-Health Research Centre, CSIROLicensed as Creative Commons – attribution, no derivatives.tag:theconversation.com,2011:article/563422016-03-20T19:27:58Z2016-03-20T19:27:58ZExplainer: Go and the ‘conversation of hands’<figure><img src="https://images.theconversation.com/files/115550/original/image-20160318-16336-1wl7ca9.png?ixlib=rb-1.1.0&q=45&auto=format&w=496&fit=clip" /><figcaption><span class="caption">Go is a beautiful and complex game that's endured for thousands of years. </span> <span class="attribution"><span class="source">Alexandre Keledjian</span>, <a class="license" href="http://creativecommons.org/licenses/by-nc/4.0/">CC BY-NC</a></span></figcaption></figure><p>Artificial intelligence reached a new frontier last week, when an AI defeated human Go champion Lee Se Dol <a href="https://gogameguru.com/alphago-defeats-lee-sedol-4-1/">four games to one</a>. </p>
<p>Google’s <a href="https://deepmind.com/alpha-go.html">Alpha Go</a> has made headlines for its ability to carry out the <a href="https://theconversation.com/ai-has-beaten-us-at-go-so-what-next-for-humanity-55945">complex calculations involved in the ancient Chinese game</a>, but I would like to give a different perspective. I want to talk about Go itself – an ancient game also known as baduk in Korean, weiqi in Chinese and Igo in Japanese - which ends, each time, with a beautiful representation of the player’s thoughts and strategies laid out across the board. </p>
<p>Go starts with an empty board of 19x19 squares. Two players take turns to place black or white stones anywhere on it, trying to surround a larger percentage of the board with their stones, or to limit the moves of the other player. </p>
<p>No stones are moved throughout the game, except when they are “captured,” by being surrounded. The aim of the game is to create spaces and connectedness. Go ends naturally, when both players agree there are no more useful moves to be made. </p>
<figure class="align-center zoomable">
<a href="https://images.theconversation.com/files/115564/original/image-20160318-16324-1hxv6b7.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=1000&fit=clip"><img alt="" src="https://images.theconversation.com/files/115564/original/image-20160318-16324-1hxv6b7.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=754&fit=clip" srcset="https://images.theconversation.com/files/115564/original/image-20160318-16324-1hxv6b7.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=600&h=400&fit=crop&dpr=1 600w, https://images.theconversation.com/files/115564/original/image-20160318-16324-1hxv6b7.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=600&h=400&fit=crop&dpr=2 1200w, https://images.theconversation.com/files/115564/original/image-20160318-16324-1hxv6b7.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=600&h=400&fit=crop&dpr=3 1800w, https://images.theconversation.com/files/115564/original/image-20160318-16324-1hxv6b7.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=754&h=503&fit=crop&dpr=1 754w, https://images.theconversation.com/files/115564/original/image-20160318-16324-1hxv6b7.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=754&h=503&fit=crop&dpr=2 1508w, https://images.theconversation.com/files/115564/original/image-20160318-16324-1hxv6b7.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=754&h=503&fit=crop&dpr=3 2262w" sizes="(min-width: 1466px) 754px, (max-width: 599px) 100vw, (min-width: 600px) 600px, 237px"></a>
<figcaption>
<span class="caption"></span>
<span class="attribution"><span class="source">2benny/Flickr</span>, <a class="license" href="http://creativecommons.org/licenses/by/4.0/">CC BY</a></span>
</figcaption>
</figure>
<p>The point of the game is not destroy your opponent, but to win with a small margin of points. It’s said that if a player is losing by more than eight points, they should resign. </p>
<p>Still, Go has never been about winning. Rather, it is about being able to develop oneself and learn. Perhaps this is why Go only made it to the West recently and instead chess, a game which is essentially destructive, has gained much more attention. </p>
<p>Go is equitable and deeply strategic, because each stone is equally valuable. The only thing that distinguishes a stone is the way it is placed at any given time. All have the potential to change the game.</p>
<p>Go derives from the Japanese word Igo. Although the game originated in China somewhere between three and five thousand years ago, it became known as Go during the <a href="http://www.britannica.com/event/Tokugawa-period">Edo period</a> (16th-19th centuries), when Japan established highly regarded and competitive schools and academies.</p>
<p>Although Japan has attracted and fostered world Go champions, such as the legendary father of the 20th century game, <a href="https://gogameguru.com/go-seigen/">Go Seigen</a>, Go has flourished throughout Asia. </p>
<p>The ethics of Go are deeply embedded in the Taoist and Confucius philosophies of self-mastery and the connection between humans and the natural environment. </p>
<p>Natural objects, such as stones and mountains, are attributed rights to exist regardless of the value they bring to the human sphere. Thus each tree or stone is intrinsically valued. </p>
<p>The four directions of the world symbolise the four sides of the Go board. The number of cross points on the board are equivalent to the number of lunar days in a year and the star points represent the most advantageous points on the board (the Goban).</p>
<figure class="align-right ">
<img alt="" src="https://images.theconversation.com/files/115557/original/image-20160318-16319-1du4hf2.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=237&fit=clip" srcset="https://images.theconversation.com/files/115557/original/image-20160318-16319-1du4hf2.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=600&h=600&fit=crop&dpr=1 600w, https://images.theconversation.com/files/115557/original/image-20160318-16319-1du4hf2.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=600&h=600&fit=crop&dpr=2 1200w, https://images.theconversation.com/files/115557/original/image-20160318-16319-1du4hf2.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=600&h=600&fit=crop&dpr=3 1800w, https://images.theconversation.com/files/115557/original/image-20160318-16319-1du4hf2.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=754&h=754&fit=crop&dpr=1 754w, https://images.theconversation.com/files/115557/original/image-20160318-16319-1du4hf2.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=754&h=754&fit=crop&dpr=2 1508w, https://images.theconversation.com/files/115557/original/image-20160318-16319-1du4hf2.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=754&h=754&fit=crop&dpr=3 2262w" sizes="(min-width: 1466px) 754px, (max-width: 599px) 100vw, (min-width: 600px) 600px, 237px">
<figcaption>
<span class="caption">Star points marked out on a standard Go board.</span>
<span class="attribution"><span class="source">Rommel2 via Wikimedia Commons</span></span>
</figcaption>
</figure>
<p>Go has been used to inform strategic decisions in governance and business. On a personal note, I used weekly games of Go to provide a conceptual framework for my PhD, by using the philosophical terms of space, connectedness and territoriality to describe the outcomes of civic engagement of recent migrants in Western Australia. The game has a supreme ability to challenge one intellectually, whilst remaining <a href="http://go-centre.nl/wp/forget-all-sorrows/">playful</a>. </p>
<p>The DeepMind challenge was not a competition, but a conversation between humans and non-humans. In the same way, Go is regarded as a <a href="http://search.proquest.com/openview/02116c59bc5b5ff32daf802d0dd6f07a/1?pq-origsite=gscholar">conversation of hands</a>. </p>
<p>When you ask for a game, you are asking “please teach me”. The player with opposite coloured stones is not only your opponent, but also your teacher and friend.</p>
<p>So let’s not forget in the debate about Google Artificial Intelligence that Go is foremost a game to be enjoyed. Much like life itself.</p><img src="https://counter.theconversation.com/content/56342/count.gif" alt="The Conversation" width="1" height="1" />
<p class="fine-print"><em><span>Silvia Lozeva does not work for, consult, own shares in or receive funding from any company or organisation that would benefit from this article, and has disclosed no relevant affiliations beyond their academic appointment.</span></em></p>An artificial intelligence has defeated a world champion of Go, the ancient Chinese strategy game. But what is Go, and why is it worth teaching to a computer?Silvia Lozeva, Researcher and Lecturer , Curtin UniversityLicensed as Creative Commons – attribution, no derivatives.tag:theconversation.com,2011:article/562092016-03-18T02:17:19Z2016-03-18T02:17:19ZGoogle’s Go victory shows AI thinking can be unpredictable, and that’s a concern<p>Humans have been taking a beating from computers lately. The <a href="http://www.abc.net.au/news/2016-03-15/google-ai-alphago-gets-divine-go-ranking/7249256">4-1 defeat</a> of Go grandmaster Lee Se-Dol by Google’s <a href="https://deepmind.com/alpha-go.html">AlphaGo</a> artificial intelligence (<a href="https://theconversation.com/au/topics/artificial-intelligence">AI</a>) is only the latest in a string of pursuits in which technology has triumphed over humanity.</p>
<p>Self-driving cars are already <a href="http://www.reuters.com/article/us-autos-alphabet-crashes-idUSKBN0UM27V20160108">less accident-prone than human drivers</a>, the TV quiz show <a href="https://www.youtube.com/watch?v=WFR3lOm_xhE">Jeopardy!</a> is a lost cause, and in chess humans have fallen so woefully behind computers that a recent international tournament <a href="http://www.hiarcs.com/Games/Mercosur2009/mercosur09.htm">was won by a mobile phone</a>. </p>
<p>There is a real sense that this month’s human vs AI Go match <a href="http://www.wired.com/2016/03/sadness-beauty-watching-googles-ai-play-go/">marks a turning point</a>. Go has long been held up as requiring levels of human intuition and pattern recognition that should be beyond the powers of number-crunching computers.</p>
<p>AlphaGo’s win over one of the world’s best players has reignited fears over the pervasive application of deep learning and AI in our future – fears famously expressed by Elon Musk as “<a href="https://www.washingtonpost.com/news/innovations/wp/2014/10/24/elon-musk-with-artificial-intelligence-we-are-summoning-the-demon/">our greatest existential threat</a>”.</p>
<p>We should consider AI a threat for two reasons, but there are approaches we can take to minimise that threat.</p>
<p>The first problem is that AI is often trained using a combination of logic and heuristics, and reinforcement learning. </p>
<p>The logic and heuristics part has reasonably predictable results: we program the rules of the game or problem into the computer, as well as some human-expert guidelines, and then use the computer’s number-crunching power to think further ahead than humans can.</p>
<p>This is how the early chess programs worked. While they played ugly chess, it was sufficient to win.</p>
<h2>The problem of reinforcement learning</h2>
<p>Reinforcement learning, on the other hand, is more opaque.</p>
<p>We have the computer perform the task – playing Go, for example – repetitively. It tweaks its strategy each time and learns the best moves from the outcomes of its play.</p>
<p>In order not to have to play humans exhaustively, this is done by playing the computer against itself. AlphaGo has played millions of games of Go – far more than any human ever has. </p>
<p>The problem is the AI will explore the entire space of possible moves and strategies in a way humans never would, and we have no insight into the methods it will derive from that exploration.</p>
<p>In the second game between Lee Se-Dol and AlphaGo, the AI made a move so surprising – “<a href="http://www.wired.com/2016/03/sadness-beauty-watching-googles-ai-play-go/">not a human move</a>” in the words of a commentator – that Lee Se-Dol had to leave the room for 15 minutes to recover his composure.</p>
<p>This is a characteristic of machine learning. The machine is not constrained by human experience or expectations.</p>
<p>Until we see an AI do the utterly unexpected, we don’t even realise that we had a limited view of the possibilities. AIs move effortlessly beyond the limits of human imagination.</p>
<p>In real-world applications, the scope for AI surprises is much wider. A stock-trading AI, for example, will re-invent every single method known to us for maximising return on investment. It will find several that are not yet known to us.</p>
<p>Unfortunately, many methods for maximising stock returns – bid support, co-ordinated trading, and so on – are regarded as illegal and unethical price manipulation.</p>
<p>How do you prevent an AI from using such methods when you don’t actually know what its methods are? Especially when the method it’s using, while unethical, may be undiscovered by human traders – literally, unknown to humankind?</p>
<p>It’s farcical to think that we will be able to predict or manage the worst-case behaviour of AIs when we can’t actually imagine their probable behaviour.</p>
<h2>The problem of ethics</h2>
<p>This leads us to the second problem. Even quite simple AIs will need to behave ethically and morally, if only to keep their operators out of jail.</p>
<p>Unfortunately, ethics and morality are not reducible to heuristics or rules.</p>
<p>Consider <a href="http://www.theatlantic.com/technology/archive/2015/10/trolley-problem-history-psychology-morality-driverless-cars/409732/">Philippa Foot’s famous trolley problem</a>:</p>
<blockquote>
<p>A trolley is running out of control down a track. In its path are five people who have been tied to the track by a mad philosopher.</p>
<p>Fortunately, you could flip a switch, which will lead the trolley down a different track to safety. Unfortunately, there is a single person tied to that track.</p>
<p>Should you flip the switch or do nothing?</p>
</blockquote>
<p>What would you expect – or instruct – an AI to do?</p>
<p><a href="http://leeds-faculty.colorado.edu/mcgrawp/PDF/BartelsPizarro.2011.pdf">In some psychological studies on the trolley problem</a>, the humans who choose to flip the switch have been found to have underlying emotional deficits and score higher on measures of psychopathy – defined in this case as “a personality style characterised by low empathy, callous affect and thrill-seeking”.</p>
<p>This suggests an important guideline for dealing with AIs. We need to understand and internalise that no matter how well they imitate or outperform humans, they will never have the intrinsic empathy or morality that causes human subjects to opt not to flip the switch. </p>
<p>Morality suggests to us that we may not take an innocent life, even when that path results in the greatest good for the greatest number.</p>
<p>Like sociopaths and psychopaths, AIs may be able to learn to imitate empathetic and ethical behaviour, but we should not expect there to be any moral force underpinning this behaviour, or that it will hold out against a purely utilitarian decision.</p>
<p>A really good rule for the use of AIs would be: “Would I put a sociopathic genius in charge of this process?” </p>
<p>There are two parts to this rule. We characterise AIs as sociopathic, in the sense of not having any genuine moral or empathetic constraints. And we characterise them as geniuses, and therefore capable of actions that we cannot foresee.</p>
<p>Playing chess and Go? Maybe. Trading on the stock market? Well, one Swiss study found <a href="http://www.forbes.com/sites/chrisbarth/2011/09/26/new-study-old-news-stock-traders-are-psychopaths/#5a6093fc1b6d">stock market traders display similarities to certified psychopaths</a>, although that’s not supposed to be a good thing. </p>
<p>But would you want an AI to look after your grandma, or to be in charge of a Predator drone?</p>
<p>There are good reasons why there is intense debate about the necessity for <a href="http://www.harvard-jlpp.com/wp-content/uploads/2013/05/36_3_1139_Marra_McNeil.pdf">a human in the loop in autonomous warfare systems</a>, but we should not be blinded to the potential for disaster in less obviously dangerous domains in which AIs are going to be deployed.</p><img src="https://counter.theconversation.com/content/56209/count.gif" alt="The Conversation" width="1" height="1" />
<p class="fine-print"><em><span>Jonathan Tapson does not work for, consult, own shares in or receive funding from any company or organisation that would benefit from this article, and has disclosed no relevant affiliations beyond their academic appointment.</span></em></p>Google’s artificial intelligence made a surprise move in the recent Go challenge that has some people worried about what happens when AI makes a non-human decision that we could not anticipate.Jonathan Tapson, Director of the MARCS Institute for Brain, Behaviour & Development, Western Sydney UniversityLicensed as Creative Commons – attribution, no derivatives.tag:theconversation.com,2011:article/559452016-03-10T19:21:09Z2016-03-10T19:21:09ZAI has beaten us at Go. So what next for humanity?<p>In the next few days, humanity’s ego is likely to take another hit when the world champion of the ancient Chinese game <a href="http://www.intergofed.org/about-go/about-go.html">Go</a> is beaten by a computer.</p>
<p>Currently Lee Sedol – the Roger Federer of Go – has lost two matches to Google’s <a href="https://deepmind.com/alpha-go.html">AlphaGo</a> program in their best-of-five series. If AlphaGo wins just one more of the remaining three matches, humanity will again be vanquished.</p>
<h2>Computer champions</h2>
<p>Back in 1979, the newly crowned world champion of backgammon, Luigi Villa, lost to the <a href="http://www.bkgm.com/articles/Berliner/BackgammonProgramBeatsWorldChamp/">BKG 9.8</a> program seven games to one in a challenge match in Monte Carlo.</p>
<p>In 1994, the <a href="https://webdocs.cs.ualberta.ca/%7Echinook/">Chinook</a> program was declared “Man-Machine World Champion” at checkers in a match against the legendary world champion Marion Tinsley after six drawn games. Sadly, Tinsley had to withdraw due to pancreatic cancer and died the following year. </p>
<p>Any doubt about the superiority of machines over humans at checkers was settled in 2007, when the developers of Chinook used a network of computers to explore the 500 billion billion possible positions and prove mathematically that a machine could play perfectly and never lose.</p>
<p>In 1997, chess fell when IBM’s <a href="https://en.wikipedia.org/wiki/Deep_Blue_versus_Garry_Kasparov">Deep Blue beat the reigning world chess champion</a>, Gary Kasparov. </p>
<p>Kasparov is generally reckoned to be one of the greatest chess players of all time. It was his sad fate that he was world champion when computing power and AI algorithms reached the point where humans were no longer able to beat machines.</p>
<h2>The ancient Chinese game of Go</h2>
<p>Go represents a significant challenge beyond chess. It’s a simple game with enormous complexity. Two players take turns to play black or white stones on a 19 by 19 board, trying to surround each other. </p>
<p>In chess, there are about 20 possible moves to consider at each turn. In Go, there are around 200. Looking just 15 black and white stones ahead involves more possible outcomes than there are atoms in the universe. </p>
<p>Another aspect of Go makes it a great challenge. In chess, it’s not too hard to work out who is winning. Just counting the value of the different pieces is a good first approximation. </p>
<p>In Go, there are just black and white stones. It takes Go masters a lifetime of training to learn when one player is ahead. </p>
<p>And any good Go program needs to work out who is ahead when deciding which of those 200 different moves to make.</p>
<figure class="align-center zoomable">
<a href="https://images.theconversation.com/files/114594/original/image-20160310-31867-ivdtan.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=1000&fit=clip"><img alt="" src="https://images.theconversation.com/files/114594/original/image-20160310-31867-ivdtan.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=754&fit=clip" srcset="https://images.theconversation.com/files/114594/original/image-20160310-31867-ivdtan.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=600&h=400&fit=crop&dpr=1 600w, https://images.theconversation.com/files/114594/original/image-20160310-31867-ivdtan.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=600&h=400&fit=crop&dpr=2 1200w, https://images.theconversation.com/files/114594/original/image-20160310-31867-ivdtan.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=600&h=400&fit=crop&dpr=3 1800w, https://images.theconversation.com/files/114594/original/image-20160310-31867-ivdtan.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=754&h=503&fit=crop&dpr=1 754w, https://images.theconversation.com/files/114594/original/image-20160310-31867-ivdtan.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=754&h=503&fit=crop&dpr=2 1508w, https://images.theconversation.com/files/114594/original/image-20160310-31867-ivdtan.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=754&h=503&fit=crop&dpr=3 2262w" sizes="(min-width: 1466px) 754px, (max-width: 599px) 100vw, (min-width: 600px) 600px, 237px"></a>
<figcaption>
<span class="caption">Go is a famously complex game.</span>
<span class="attribution"><span class="source">Linh Nguyen/Flickr</span>, <a class="license" href="http://creativecommons.org/licenses/by-nc-nd/4.0/">CC BY-NC-ND</a></span>
</figcaption>
</figure>
<h2>AlphaGo’s secrets</h2>
<p>Google’s AlphaGo uses an elegant marriage of computer brute force and human-style perception to tackle these two problems. </p>
<p>To deal with the immense size of the <a href="https://en.wikipedia.org/wiki/Game_tree">game tree</a> – which represents the various possible moves by each player – AlphaGo uses an AI heuristic called <a href="https://en.wikipedia.org/wiki/Monte_Carlo_tree_search">Monte Carlo tree search</a>, where the computer uses its grunt to explore a random sample of the possible moves. </p>
<p>On the other hand, to deal with the difficulty of recognising who is ahead, AlphaGo uses a fashionable machine learning technique called “<a href="https://developer.nvidia.com/deep-learning">deep learning</a>”. </p>
<p>The computer is shown a huge database of past games. It then plays itself millions and millions of times in order to match, and ultimately exceed, a Go master’s ability to decide who is ahead.</p>
<p>Less discussed are the returns gained from Google’s engineering expertise and vast server farms. Like a lot of recent advances in AI, a significant return has come from throwing many more resources at the problem. </p>
<p>Before AlphaGo, computer Go programs were mostly the efforts of a single person run on just one computer. But AlphaGo represents a significant engineering effort from dozens and dozens of Google’s engineers and top AI scientists, as well as the benefits of access to Google’s server farms. </p>
<h2>What next?</h2>
<p>Beating humans at this very challenging board game is certainly a landmark moment. I am not sure that I agree with <a href="http://demishassabis.com/biography/">Demis Hassabis</a>, the leader of the AlphaGo project, that Go is “<a href="http://www.theguardian.com/technology/2016/feb/16/demis-hassabis-artificial-intelligence-deepmind-alphago?via=indexdotco">the pinnacle of games, and the richest in terms of intellectual depth</a>”. </p>
<p>It is certainly the Mount Everest as it has the largest game tree. However, a game like poker is the K2, as it introduces a number of additional factors like uncertainty of where the cards lie and the psychology of your opponents. This makes it arguably a greater intellectual challenge.</p>
<p>And despite the claims that the methods used to solve Go are general purpose, it would take a significant human effort to get AlphaGo to play a game like chess well.</p>
<p>Nevertheless, the ideas and AI techniques that went into AlphaGo are likely to find their way into new applications soon. And it won’t be just in games. We’ll seen them in areas like Google’s page ranking, adwords, speech recognition and even driverless cars.</p>
<h2>Our machine overlords</h2>
<p>You don’t have to worry that computers will be lording it over us any time soon. AlphaGo has no autonomy. It has no desires other than to play Go. </p>
<p>It won’t wake up tomorrow and realise it’s bored of Go and decide to win some money at poker. Or that it wants to take over the world.</p>
<p>But it does represent another specialised task where machine is now better than human. </p>
<p>This is where the real challenge is coming. What do we do when some of our specialised skills – playing Go, writing newspaper articles, or driving cars – are automated?</p><img src="https://counter.theconversation.com/content/55945/count.gif" alt="The Conversation" width="1" height="1" />
<p class="fine-print"><em><span>Toby Walsh does not work for, consult, own shares in or receive funding from any company or organisation that would benefit from this article, and has disclosed no relevant affiliations beyond their academic appointment.</span></em></p>A machine has bested us at yet another intellectually challenging game. It shows artificial intelligence is progressing rapidly, but it doesn’t mean humans are redundant quite yet.Toby Walsh, Professor of AI at UNSW, Research Group Leader at Data61., Data61Licensed as Creative Commons – attribution, no derivatives.tag:theconversation.com,2011:article/541002016-02-05T11:08:55Z2016-02-05T11:08:55ZEvolving our way to artificial intelligence<figure><img src="https://images.theconversation.com/files/110042/original/image-20160202-32254-zn1mf2.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=496&fit=clip" /><figcaption><span class="caption">Just Go for it: programming a computer to play an ancient game.</span> <span class="attribution"><span class="source">Donar Reiskoffer/Wikimedia Commons</span>, <a class="license" href="http://creativecommons.org/licenses/by-sa/4.0/">CC BY-SA</a></span></figcaption></figure><p>Researcher David Silver and colleagues designed a computer program <a href="http://www.nature.com/nature/journal/v529/n7587/full/nature16961.html">capable of beating a top-level Go player</a> – a marvelous technological feat and important threshold in the development of artificial intelligence, or AI. It stresses once more that humans aren’t at the center of the universe, and that human cognition isn’t the pinnacle of intelligence.</p>
<p>I remember well when IBM’s computer <a href="http://www.nybooks.com/articles/2010/02/11/the-chess-master-and-the-computer/">Deep Blue beat chess master Garry Kasparov</a>. Where I’d played – and lost to – chess-playing computers myself, the Kasparov defeat solidified my personal belief that artificial intelligence will become reality, probably even in my lifetime. I might one day be able to talk to things similar to my childhood heroes C-3PO and R2-D2. My future house could be controlled by a program like HAL from Kubrick’s “2001” movie. </p>
<figure>
<iframe width="440" height="260" src="https://www.youtube.com/embed/ARJ8cAGm6JE?wmode=transparent&start=0" frameborder="0" allowfullscreen=""></iframe>
<figcaption><span class="caption">Not the best automated-home controller: HAL.</span></figcaption>
</figure>
<p>As a researcher in artificial intelligence, I realize how impressive it is to have a computer beat a top Go player, a much tougher technical challenge than winning at chess. Yet it’s still not a big step toward the type of artificial intelligence used by the thinking machines we see in the movies. For that, we need new approaches to developing AI.</p>
<h2>Intelligence is evolved, not engineered</h2>
<p>To understand the limitations of the Go milestone, we need to think about what artificial intelligence is – and how the research community makes progress in the field.</p>
<p>Typically, AI is part of the domain of engineering and computer science, a field in which progress is measured not by how much we learned about nature or humans, but by achieving a well-defined goal: if the bridge can carry a 120-ton truck, it succeeds. Beating a human at Go falls into exactly that category.</p>
<p>I take a different approach. When I talk about AI, I typically don’t talk about a well-defined matter. Rather, I describe the AI that I would like to have as “a machine that has cognitive abilities comparable to that of a human.”</p>
<p>Admittedly, that is a very fuzzy goal, but that is the whole point. We can’t engineer what we can’t define, which is why I think the engineering approach to “human level cognition” – that is, writing smart algorithms to solve a particularly well-defined problem – isn’t going to get us where we want to go. But then what is? </p>
<p>We can’t wait for cognitive- and neuroscience, behavior biology or psychology to figure out what the brain does and how it works. Even if we wait, these sciences will not come up with a simple algorithm explaining the human brain. </p>
<p>What we do know is that the brain wasn’t engineered with a simple modular building plan in mind. It was cobbled together by Darwinian evolution – an opportunistic mechanism governed by the simple rule that whoever makes more viable offspring wins the race.</p>
<p>This explains why I work on the <a href="http://hintzelab.msu.edu/">evolution of artificial intelligence</a> and try to understand the evolution of natural intelligence. I make a living out of <a href="http://phenomena.nationalgeographic.com/2013/08/02/meet-the-animats/">evolving digital brains</a>.</p>
<figure class="align-center zoomable">
<a href="https://images.theconversation.com/files/110349/original/image-20160204-3027-cmklb3.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=1000&fit=clip"><img alt="" src="https://images.theconversation.com/files/110349/original/image-20160204-3027-cmklb3.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=754&fit=clip" srcset="https://images.theconversation.com/files/110349/original/image-20160204-3027-cmklb3.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=600&h=214&fit=crop&dpr=1 600w, https://images.theconversation.com/files/110349/original/image-20160204-3027-cmklb3.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=600&h=214&fit=crop&dpr=2 1200w, https://images.theconversation.com/files/110349/original/image-20160204-3027-cmklb3.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=600&h=214&fit=crop&dpr=3 1800w, https://images.theconversation.com/files/110349/original/image-20160204-3027-cmklb3.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=754&h=269&fit=crop&dpr=1 754w, https://images.theconversation.com/files/110349/original/image-20160204-3027-cmklb3.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=754&h=269&fit=crop&dpr=2 1508w, https://images.theconversation.com/files/110349/original/image-20160204-3027-cmklb3.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=754&h=269&fit=crop&dpr=3 2262w" sizes="(min-width: 1466px) 754px, (max-width: 599px) 100vw, (min-width: 600px) 600px, 237px"></a>
<figcaption>
<span class="caption">Divergent evolution: These two figures show maps of different evolutions of connections between digital brain parts, 49,000 generations after they both began at the same starting point.</span>
<span class="attribution"><a class="source" href="http://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1002236">Arend Hintze</a>, <a class="license" href="http://creativecommons.org/licenses/by/4.0/">CC BY</a></span>
</figcaption>
</figure>
<h2>Algorithms vs. improvisation</h2>
<p>To return to the Go algorithm: in the context of computer games, improving skill is possible only by playing against a better competitor. </p>
<p>The Go victory shows that we can make better algorithms for more complex problems than before. That in turn suggests that in the future, we could see more computer games with complex rules providing better opponent AI against human players. <a href="http://www.bbc.com/future/story/20151201-the-cyborg-chess-players-that-cant-be-beaten">Chess computers</a> have changed how modern chess is played, and we can expect a similar effect for Go and its players.</p>
<p>This new algorithm provides a way to define optimal play, which is probably good if you want to learn Go or improve your skills. However, since this new algorithm is pretty much the best possible Go player on Earth, playing against it nearly guarantees you’ll lose. That’s no fun.</p>
<p>Fortunately, continuous loss doesn’t have to happen. The computer’s controllers can make the algorithm play less well by either reducing the number of moves it thinks ahead, or – and this is really new – using a less-developed <a href="http://www.nature.com/news/computer-science-the-learning-machines-1.14481">deep neural net</a> to evaluate the Go board. </p>
<p>But does this make the algorithm play more like a human, and is that what we want in a Go player? Let us turn to other games that have fewer fixed rules and instead require the player to improvise more.</p>
<p>Imagine a first person shooter, or a multiplayer battle game, or a typical role-playing adventure game. These games became popular not because people could play them against better AI, but because they can be played against, or together with, other human beings.</p>
<p>It seems as if we are not necessarily looking for strength and skill in opponents we play, but for human characteristics like being able to surprise us, to see the same humor and maybe to even empathize with us. </p>
<p>For example, I recently played <a href="http://thatgamecompany.com/games/journey/">Journey</a>, a game where the only way other online players can interact with each other is by singing a particular tune that each can hear and see. This is a creative and emotional way for a player to look at the beautiful art of that game and share important moments of its story with someone else.</p>
<figure class="align-center zoomable">
<a href="https://images.theconversation.com/files/110179/original/image-20160203-5861-1o40ndq.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=1000&fit=clip"><img alt="" src="https://images.theconversation.com/files/110179/original/image-20160203-5861-1o40ndq.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=754&fit=clip" srcset="https://images.theconversation.com/files/110179/original/image-20160203-5861-1o40ndq.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=600&h=338&fit=crop&dpr=1 600w, https://images.theconversation.com/files/110179/original/image-20160203-5861-1o40ndq.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=600&h=338&fit=crop&dpr=2 1200w, https://images.theconversation.com/files/110179/original/image-20160203-5861-1o40ndq.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=600&h=338&fit=crop&dpr=3 1800w, https://images.theconversation.com/files/110179/original/image-20160203-5861-1o40ndq.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=754&h=424&fit=crop&dpr=1 754w, https://images.theconversation.com/files/110179/original/image-20160203-5861-1o40ndq.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=754&h=424&fit=crop&dpr=2 1508w, https://images.theconversation.com/files/110179/original/image-20160203-5861-1o40ndq.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=754&h=424&fit=crop&dpr=3 2262w" sizes="(min-width: 1466px) 754px, (max-width: 599px) 100vw, (min-width: 600px) 600px, 237px"></a>
<figcaption>
<span class="caption">Playing with your emotions: In the video game Journey, intercharacter connection is a key feature.</span>
<span class="attribution"><span class="source">Journey/That Game Company</span>, <a class="license" href="http://creativecommons.org/licenses/by-nd/4.0/">CC BY-ND</a></span>
</figcaption>
</figure>
<p>It is the emotional connection that makes this experience remarkable, and not the skill of the other player. </p>
<p>If the AI that controls other players <em>evolved</em>, it may go through the same steps that made our brain work. That could include sensing emotional equivalents to fear, warning about undetermined threats, and probably also empathy to understand other organisms and their needs.</p>
<p>It is this, and the AI’s ability to do different things instead of being a specialist in just one realm, that I am looking for in AI. We might, therefore, need to incorporate the process of how we became us into the process of how we make our digital counterparts.</p><img src="https://counter.theconversation.com/content/54100/count.gif" alt="The Conversation" width="1" height="1" />
<p class="fine-print"><em><span>Arend Hintze receives funding from NSF, Strength in Numbers Studios, and prior to that from the Allen Research Foundation. He works for Michigan State University and collaborates with Strength in Numbers on the evolution of Artificial Intelligence in computer games.</span></em></p>While it’s impressive, developing a computer to win at Go is not a big step toward the type of artificial intelligence used by the thinking machines we see in the movies.Arend Hintze, Assistant Professor of Integrative Biology & Computer Science and Engineering, Michigan State UniversityLicensed as Creative Commons – attribution, no derivatives.tag:theconversation.com,2011:article/537622016-01-27T18:01:54Z2016-01-27T18:01:54ZGoogle’s Go triumph is a milestone for artificial intelligence research<figure><img src="https://images.theconversation.com/files/109383/original/image-20160127-26788-1adobit.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=496&fit=clip" /><figcaption><span class="caption">
</span> <span class="attribution"><span class="source">Lyle J Hatch / shutterstock.com</span></span></figcaption></figure><p>Researchers from Google DeepMind have developed the first computer able to defeat a human champion at the board game Go. But why has the online giant invested millions of dollars and some of the finest minds in Artificial Intelligence (AI) research to create a computer board game player? </p>
<p>Go is not just any board game. It’s more than 2,000 years old and is played by more than <a href="http://www.britgo.org/press/faq.html">60m people</a> across the world – including a thousand professionals. Creating a superhuman computer Go player able to beat these top pros has been one of the most challenging targets of AI research for decades.</p>
<p>The rules are deceptively simple: two players take turns to place white and black “stones” on an empty 19x19 board, each aiming to encircle the most territory. Yet these basics yield a game of extraordinary beauty and complexity, full of patterns and flow. Go has many more <a href="http://tromp.github.io/go/legal.html">possible positions</a> than even chess – in fact, there are more possibilities in a game of Go than we would get by considering a separate chess game played on every atom in the universe.</p>
<p>AI researchers have therefore long regarded Go as a “grand challenge”. Whereas even the best human chess players had fallen to computers by the 1990s, Go remained unbeaten. This is a truly historic breakthrough. </p>
<h2>Games are the ‘lab rats’ of AI research</h2>
<p>Since the term “artificial intelligence” or “AI” was first coined in the 1950s, the range of problems which it can solve has been increasing at an accelerating rate. We take it for granted that Amazon has a pretty good idea of what we might want to buy, for instance, or that Google can complete our partially typed search term, though these are both due to <a href="http://www.wired.com/2015/04/now-anyone-can-tap-ai-behind-amazons-recommendations/">recent advances in AI</a>.</p>
<figure class="align-center ">
<img alt="" src="https://images.theconversation.com/files/109388/original/image-20160127-26823-16tq0i9.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=754&fit=clip" srcset="https://images.theconversation.com/files/109388/original/image-20160127-26823-16tq0i9.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=600&h=462&fit=crop&dpr=1 600w, https://images.theconversation.com/files/109388/original/image-20160127-26823-16tq0i9.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=600&h=462&fit=crop&dpr=2 1200w, https://images.theconversation.com/files/109388/original/image-20160127-26823-16tq0i9.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=600&h=462&fit=crop&dpr=3 1800w, https://images.theconversation.com/files/109388/original/image-20160127-26823-16tq0i9.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=754&h=580&fit=crop&dpr=1 754w, https://images.theconversation.com/files/109388/original/image-20160127-26823-16tq0i9.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=754&h=580&fit=crop&dpr=2 1508w, https://images.theconversation.com/files/109388/original/image-20160127-26823-16tq0i9.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=754&h=580&fit=crop&dpr=3 2262w" sizes="(min-width: 1466px) 754px, (max-width: 599px) 100vw, (min-width: 600px) 600px, 237px">
<figcaption>
<span class="caption">Go originated in China over 2,000 years ago and is played by millions.</span>
<span class="attribution"><a class="source" href="https://www.flickr.com/photos/adavey/4867276096/">Alan</a>, <a class="license" href="http://creativecommons.org/licenses/by/4.0/">CC BY</a></span>
</figcaption>
</figure>
<p>Computer games have been a crucial test bed for developing and testing new AI techniques – the “lab rat” of our research. This has led to superhuman players in <a href="http://science.sciencemag.org/content/317/5836/308.1.full">checkers</a>, <a href="http://blogs.gartner.com/andrew_white/2014/03/12/the-chess-master-and-the-machine-the-truth-behind-kasparov-versus-deep-blue/">chess</a>, <a href="http://aitopics.org/topic/scrabble">Scrabble</a>, <a href="http://aitopics.org/topic/backgammon">backgammon</a> and more recently, simple forms of <a href="http://www.sciencemag.org/news/2015/01/texas-hold-em-poker-solved-computer">poker</a>. </p>
<p>Games provide a fascinating source of tough problems – they have well-defined rules and a clear target: to win. To beat these games the AIs were programmed to search forward into possible futures and choose the move which leads to the best outcome – which is similar to how good human players make decisions.</p>
<p>Yet Go proved hardest to beat because of its enormous search space and the difficulty of working out who is winning from an unfinished game position. Back in 2001, Jonathan Schaeffer, a brilliant researcher who created a perfect AI checkers player, <a href="http://aaai.org/ojs/index.php/aimagazine/article/download/1570/1469">said it would</a> “take many decades of research and development before world-championship-caliber Go programs exist”. Until now, even with recent advances, it still seemed at least ten years out of reach.</p>
<h2>The breakthrough</h2>
<p>Google’s announcement, in the journal <a href="http://nature.com/articles/doi:10.1038/nature16961">Nature</a>, details
how its machine “learned” to play Go by analysing millions of past games by professional human players and simulating thousands of possible future game states per second. Specifically, the researchers at DeepMind trained “convolutional neural networks”, algorithms that mimic the high-level structure of the brain and visual system and which have recently seen <a href="http://www.wired.com/tag/deep-learning/">an explosion in their effectiveness</a>, to predict expert moves. </p>
<p>This learning was combined with <a href="http://www.mcts.ai/">Monte Carlo tree search</a> approaches which use randomness and machine learning to intelligently search the “tree” of possible future board states. These searches have massively increased the strength of computer Go players since their invention less than ten years ago, as well as finding applications in many other domains. </p>
<figure class="align-right ">
<img alt="" src="https://images.theconversation.com/files/109384/original/image-20160127-26817-1ih81mc.png?ixlib=rb-1.1.0&q=45&auto=format&w=237&fit=clip" srcset="https://images.theconversation.com/files/109384/original/image-20160127-26817-1ih81mc.png?ixlib=rb-1.1.0&q=45&auto=format&w=600&h=546&fit=crop&dpr=1 600w, https://images.theconversation.com/files/109384/original/image-20160127-26817-1ih81mc.png?ixlib=rb-1.1.0&q=30&auto=format&w=600&h=546&fit=crop&dpr=2 1200w, https://images.theconversation.com/files/109384/original/image-20160127-26817-1ih81mc.png?ixlib=rb-1.1.0&q=15&auto=format&w=600&h=546&fit=crop&dpr=3 1800w, https://images.theconversation.com/files/109384/original/image-20160127-26817-1ih81mc.png?ixlib=rb-1.1.0&q=45&auto=format&w=754&h=686&fit=crop&dpr=1 754w, https://images.theconversation.com/files/109384/original/image-20160127-26817-1ih81mc.png?ixlib=rb-1.1.0&q=30&auto=format&w=754&h=686&fit=crop&dpr=2 1508w, https://images.theconversation.com/files/109384/original/image-20160127-26817-1ih81mc.png?ixlib=rb-1.1.0&q=15&auto=format&w=754&h=686&fit=crop&dpr=3 2262w" sizes="(min-width: 1466px) 754px, (max-width: 599px) 100vw, (min-width: 600px) 600px, 237px">
<figcaption>
<span class="caption">Only human: Fan Hui at a tournament in 2006.</span>
<span class="attribution"><a class="source" href="https://www.flickr.com/photos/72563913@N00/131276951">lyonshinogi</a>, <a class="license" href="http://creativecommons.org/licenses/by-sa/4.0/">CC BY-SA</a></span>
</figcaption>
</figure>
<p>The resulting “player” significantly outperformed all existing state-of-the-art AI players and went on to beat the current European champion, Fan Hui, 5-0 under tournament conditions.</p>
<h2>AI passes ‘Go’</h2>
<p>Now that Go has seemingly been cracked, AI needs a new grand challenge – a new “lab rat” – and it seems likely that many of these challenges will come from the $100 billion digital games industry. The ability to play alongside or against millions of engaged human players provides unique opportunities for AI research. At York’s centre for <a href="http://www.iggi.org.uk">Intelligent Games and Game Intelligence</a>, we’re working on projects such as building an AI aimed at player fun (rather than playing strength), for instance, or using games to improve well-being of people with Alzheimer’s. Collaborations between multidisciplinary labs like ours, the games industry and big business are likely to yield the next big AI breakthroughs.</p>
<figure>
<iframe width="440" height="260" src="https://www.youtube.com/embed/nD0lPW-cc1g?wmode=transparent&start=0" frameborder="0" allowfullscreen=""></iframe>
<figcaption><span class="caption">A computer can run through thousands of these per second.</span></figcaption>
</figure>
<p>However the real world is a step up, full of ill-defined questions that are far more complex than even the trickiest of board games. The techniques which conquered Go can certainly be applied in <a href="http://www.ibm.com/smarterplanet/us/en/ibmwatson/health/">medicine</a>, <a href="https://www.glasslabgames.org/">education</a>, <a href="http://www.sciencedaily.com/releases/2015/11/151117092418.htm">science</a> or any other domain where data is available and outcomes can be evaluated and understood. </p>
<p>The big question is whether Google just helped us towards the next generation of <a href="http://waitbutwhy.com/2015/01/artificial-intelligence-revolution-1.html">Artificial <em>General</em> Intelligence</a>, where machines learn to truly think like – and beyond – humans. Whether we’ll see AlphaGo as a step towards Hollywood’s dreams (and nightmares) of AI agents with self-awareness, emotion and motivation remains to be seen. However the latest breakthrough points to a brave new future where AI will continue to improve our lives by helping us to make better-informed decisions in a world of ever-increasing complexity.</p><img src="https://counter.theconversation.com/content/53762/count.gif" alt="The Conversation" width="1" height="1" />
<p class="fine-print"><em><span>The authors do not work for, consult, own shares in or receive funding from any company or organisation that would benefit from this article, and have disclosed no relevant affiliations beyond their academic appointment.</span></em></p>Even the smartest AIs weren’t supposed to beat top humans at Go for another decade or more.Peter Cowling, Professor of Computer Science, University of YorkSam Devlin, Research Fellow in Artificial Intelligence and Digital Games, University of YorkLicensed as Creative Commons – attribution, no derivatives.