tag:theconversation.com,2011:/us/topics/artificial-neural-networks-72832/articlesArtificial neural networks – The Conversation2024-02-29T13:39:54Ztag:theconversation.com,2011:article/2227002024-02-29T13:39:54Z2024-02-29T13:39:54ZWe’ve been here before: AI promised humanlike machines – in 1958<figure><img src="https://images.theconversation.com/files/578758/original/file-20240228-16-mnuihk.jpg?ixlib=rb-1.1.0&rect=0%2C0%2C2048%2C1603&q=45&auto=format&w=496&fit=clip" /><figcaption><span class="caption">Frank Rosenblatt with the Mark I Perceptron, the first artificial neural network computer, unveiled in 1958.</span> <span class="attribution"><a class="source" href="https://www.flickr.com/photos/127906254@N06/20897323365/in/photolist-5VsZ1M-5Vjepm-xQCfbH-5WbkWz-5Wdtn4-5WdqXa-f2s3pc">National Museum of the U.S. Navy/Flickr</a></span></figcaption></figure><p>A roomsize computer equipped with a new type of circuitry, the Perceptron, was introduced to the world in 1958 in a <a href="https://www.nytimes.com/1958/07/08/archives/new-navy-device-learns-by-doing-psychologist-shows-embryo-of.html">brief news story</a> buried deep in The New York Times. The story cited the U.S. Navy as saying that the Perceptron would lead to machines that “will be able to walk, talk, see, write, reproduce itself and be conscious of its existence.” </p>
<p>More than six decades later, similar claims are being made about current artificial intelligence. So, what’s changed in the intervening years? In some ways, not much. </p>
<p>The field of artificial intelligence has been running through a boom-and-bust cycle since its early days. Now, as the field is in yet another boom, many proponents of the technology seem to have forgotten the failures of the past – and the reasons for them. While optimism drives progress, it’s worth paying attention to the history. </p>
<p>The Perceptron, <a href="https://psycnet.apa.org/doi/10.1037/h0042519">invented by Frank Rosenblatt</a>, arguably laid the <a href="https://news.cornell.edu/stories/2019/09/professors-perceptron-paved-way-ai-60-years-too-soon">foundations for AI</a>. The electronic analog computer was a learning machine designed to predict whether an image belonged in one of two categories. This revolutionary machine was filled with wires that physically connected different components together. Modern day artificial neural networks that underpin familiar AI like ChatGPT and DALL-E are software versions of the Perceptron, except with substantially more layers, nodes and connections.</p>
<p>Much like modern-day machine learning, if the Perceptron returned the wrong answer, it would alter its connections so that it could make a better prediction of what comes next the next time around. Familiar modern AI systems work in much the same way. Using a prediction-based format, large language models, or LLMs, are able to produce impressive <a href="https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-doing-and-why-does-it-work/">long-form text-based responses</a> and associate images with text to produce <a href="https://www.assemblyai.com/blog/how-dall-e-2-actually-works/">new images based on prompts</a>. These systems get better and better as they interact more with users. </p>
<figure class="align-center zoomable">
<a href="https://images.theconversation.com/files/576348/original/file-20240219-22-8zapyi.png?ixlib=rb-1.1.0&q=45&auto=format&w=1000&fit=clip"><img alt="A chart with a horizontal row of nine colored blocks through the center and numerous black vertical lines connecting the blocks with sections of text above and below the blocks" src="https://images.theconversation.com/files/576348/original/file-20240219-22-8zapyi.png?ixlib=rb-1.1.0&q=45&auto=format&w=754&fit=clip" srcset="https://images.theconversation.com/files/576348/original/file-20240219-22-8zapyi.png?ixlib=rb-1.1.0&q=45&auto=format&w=600&h=450&fit=crop&dpr=1 600w, https://images.theconversation.com/files/576348/original/file-20240219-22-8zapyi.png?ixlib=rb-1.1.0&q=30&auto=format&w=600&h=450&fit=crop&dpr=2 1200w, https://images.theconversation.com/files/576348/original/file-20240219-22-8zapyi.png?ixlib=rb-1.1.0&q=15&auto=format&w=600&h=450&fit=crop&dpr=3 1800w, https://images.theconversation.com/files/576348/original/file-20240219-22-8zapyi.png?ixlib=rb-1.1.0&q=45&auto=format&w=754&h=565&fit=crop&dpr=1 754w, https://images.theconversation.com/files/576348/original/file-20240219-22-8zapyi.png?ixlib=rb-1.1.0&q=30&auto=format&w=754&h=565&fit=crop&dpr=2 1508w, https://images.theconversation.com/files/576348/original/file-20240219-22-8zapyi.png?ixlib=rb-1.1.0&q=15&auto=format&w=754&h=565&fit=crop&dpr=3 2262w" sizes="(min-width: 1466px) 754px, (max-width: 599px) 100vw, (min-width: 600px) 600px, 237px"></a>
<figcaption>
<span class="caption">A timeline of the history of AI starting in the 1940s. Click the author’s name here for a PDF of this poster.</span>
<span class="attribution"><a class="source" href="https://www.daniellejwilliams.com/_files/ugd/a6ff55_cac7c8efb9404a208c0ecd284ff11ba7.pdf">Danielle J. Williams</a>, <a class="license" href="http://creativecommons.org/licenses/by-nd/4.0/">CC BY-ND</a></span>
</figcaption>
</figure>
<h2>AI boom and bust</h2>
<p>In the decade or so after Rosenblatt unveiled the Mark I Perceptron, experts like <a href="https://www.nytimes.com/2016/01/26/business/marvin-minsky-pioneer-in-artificial-intelligence-dies-at-88.html">Marvin Minsky</a> claimed that the world would “<a href="https://books.google.com/books?id=2FMEAAAAMBAJ&pg=PA58&dq=In+from+three+to+eight+years+we+will+have+a+machine+with+the+general+intelligence+of+an+average+human+being#v=onepage&q=In%20from%20three%20to%20eight%20years%20we%20will%20have%20a%20machine%20with%20the%20general%20intelligence%20of%20an%20average%20human%20being&f=false">have a machine with the general intelligence of an average human being</a>” by the mid- to late-1970s. But despite some success, humanlike intelligence was nowhere to be found. </p>
<p>It quickly became apparent that the <a href="https://stacks.stanford.edu/file/druid:cn981xh0967/cn981xh0967.pdf">AI systems knew nothing about their subject matter</a>. Without the appropriate background and contextual knowledge, it’s nearly impossible to accurately resolve ambiguities present in everyday language – a task humans perform effortlessly. The first AI “winter,” or period of disillusionment, hit in 1974 following the <a href="https://dougenterprises.com/perceptron-history/">perceived failure of the Perceptron</a>.</p>
<p>However, by 1980, AI was back in business, and the first official AI boom was in full swing. There were new <a href="https://www.britannica.com/technology/expert-system">expert systems</a>, AIs designed to solve problems in specific areas of knowledge, that could identify objects and <a href="https://www.britannica.com/technology/MYCIN">diagnose diseases from observable data</a>. There were programs that could make <a href="https://eric.ed.gov/?id=ED161024">complex inferences from simple stories</a>, the <a href="https://web.stanford.edu/%7Elearnest/sail/oldcart.html">first driverless car</a> was ready to hit the road, and <a href="https://robotsguide.com/robots/wabot">robots that could read and play music</a> were playing for live audiences. </p>
<p>But it wasn’t long before the same problems stifled excitement once again. In 1987, the second AI winter hit. Expert systems were failing because <a href="https://towardsdatascience.com/history-of-the-second-ai-winter-406f18789d45">they couldn’t handle novel information</a>. </p>
<p>The 1990s changed the way experts approached problems in AI. Although the eventual thaw of the second winter didn’t lead to an official boom, AI underwent substantial changes. Researchers were tackling the <a href="https://doi.org/10.1145/97709.97728">problem of knowledge acquisition</a> with <a href="https://www.lightsondata.com/the-history-of-machine-learning/#:%7E:text=In%20the%201990s%20work%20on,learn%E2%80%9D%20%E2%80%94%20from%20the%20results.">data-driven approaches</a> to machine learning that changed how AI acquired knowledge.</p>
<p>This time also marked a return to the neural-network-style perceptron, but this version was far more complex, dynamic and, most importantly, digital. The return to the neural network, along with the invention of the web browser and an increase in computing power, <a href="https://www.analyticsvidhya.com/blog/2020/09/quick-history-neural-networks/">made it easier to collect images, mine for data and distribute datasets for machine learning tasks</a>. </p>
<h2>Familiar refrains</h2>
<p>Fast forward to today and confidence in AI progress has begun once again to echo promises made nearly 60 years ago. The term “<a href="https://www.ibm.com/topics/strong-ai">artificial general intelligence</a>” is used to describe the activities of LLMs like those powering AI chatbots like ChatGPT. Artificial general intelligence, or AGI, describes a machine that has intelligence equal to humans, meaning the machine would be self-aware, able to solve problems, learn, plan for the future and possibly be conscious. </p>
<p>Just as Rosenblatt thought his Perceptron was a foundation for a conscious, humanlike machine, so do some contemporary AI theorists about today’s artificial neural networks. In 2023, Microsoft published a paper saying that “<a href="https://doi.org/10.48550/arXiv.2303.12712">GPT-4’s performance is strikingly close to human-level performance</a>.” </p>
<figure class="align-center zoomable">
<a href="https://images.theconversation.com/files/578773/original/file-20240228-30-172pg4.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=1000&fit=clip"><img alt="Three men sit in chairs on a stage" src="https://images.theconversation.com/files/578773/original/file-20240228-30-172pg4.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=754&fit=clip" srcset="https://images.theconversation.com/files/578773/original/file-20240228-30-172pg4.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=600&h=400&fit=crop&dpr=1 600w, https://images.theconversation.com/files/578773/original/file-20240228-30-172pg4.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=600&h=400&fit=crop&dpr=2 1200w, https://images.theconversation.com/files/578773/original/file-20240228-30-172pg4.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=600&h=400&fit=crop&dpr=3 1800w, https://images.theconversation.com/files/578773/original/file-20240228-30-172pg4.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=754&h=503&fit=crop&dpr=1 754w, https://images.theconversation.com/files/578773/original/file-20240228-30-172pg4.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=754&h=503&fit=crop&dpr=2 1508w, https://images.theconversation.com/files/578773/original/file-20240228-30-172pg4.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=754&h=503&fit=crop&dpr=3 2262w" sizes="(min-width: 1466px) 754px, (max-width: 599px) 100vw, (min-width: 600px) 600px, 237px"></a>
<figcaption>
<span class="caption">Executives at big tech companies, including Meta, Google and OpenAI, have set their sights on developing human-level AI.</span>
<span class="attribution"><a class="source" href="https://newsroom.ap.org/detail/APECFutureofAI/3fd286588bd549f196eeed9b3c6919fe/photo?Query=Sam%20Altman&mediaType=photo&sortBy=creationdatetime:desc&dateRange=Anytime&totalCount=164&currentItemNo=38">AP Photo/Eric Risberg</a></span>
</figcaption>
</figure>
<p>But before claiming that LLMs are exhibiting human-level intelligence, it might help to reflect on the cyclical nature of AI progress. Many of the same problems that haunted earlier iterations of AI are still present today. The difference is how those problems manifest. </p>
<p>For example, the knowledge problem persists to this day. ChatGPT continually struggles to respond to <a href="https://blogs.nottingham.ac.uk/makingsciencepublic/2023/10/27/chatgpt-and-its-magical-metaphors/">idioms, metaphors, rhetorical questions and sarcasm</a> – unique forms of language that go beyond grammatical connections and instead require inferring the meaning of the words based on context. </p>
<p>Artificial neural networks can, with impressive accuracy, pick out objects in complex scenes. But give an AI a picture of a school bus lying on its side and it will very confidently <a href="https://arxiv.org/abs/1811.11553">say it’s a snowplow</a> 97% of the time. </p>
<h2>Lessons to heed</h2>
<p>In fact, it turns out that AI is <a href="https://www.nature.com/articles/d41586-019-03013-5">quite easy to fool</a> in ways that humans would immediately identify. I think it’s a consideration worth taking seriously in light of how things have gone in the past.</p>
<p>The AI of today looks quite different than AI once did, but the problems of the past remain. As the saying goes: History may not repeat itself, but it often rhymes.</p><img src="https://counter.theconversation.com/content/222700/count.gif" alt="The Conversation" width="1" height="1" />
<p class="fine-print"><em><span>Danielle Williams does not work for, consult, own shares in or receive funding from any company or organization that would benefit from this article, and has disclosed no relevant affiliations beyond their academic appointment.</span></em></p>Enthusiasm for the capabilities of artificial intelligence – and claims for the approach of humanlike prowess –has followed a boom-and-bust cycle since the middle of the 20th century.Danielle Williams, Postdoctoral Fellow in Philosophy of Science, Arts & Sciences at Washington University in St. LouisLicensed as Creative Commons – attribution, no derivatives.tag:theconversation.com,2011:article/2234882024-02-16T16:07:23Z2024-02-16T16:07:23ZAI has a large and growing carbon footprint, but there are potential solutions on the horizon<figure><img src="https://images.theconversation.com/files/575971/original/file-20240215-26-d8wa2k.jpg?ixlib=rb-1.1.0&rect=0%2C0%2C3834%2C2160&q=45&auto=format&w=496&fit=clip" /><figcaption><span class="caption">
</span> <span class="attribution"><a class="source" href="https://www.shutterstock.com/image-photo/futuristic-concept-data-center-chief-technology-2200880615">Gorodenkoff / Shutterstock</a></span></figcaption></figure><p>Given the huge problem-solving potential of artificial intelligence (AI), it wouldn’t be far-fetched to think that AI could also help us in <a href="https://theconversation.com/four-ways-ai-could-help-us-respond-to-climate-change-despite-how-much-energy-it-uses-208135">tackling the climate crisis</a>. However, when we consider the energy needs of AI models, it becomes clear that the technology is as much a part of the climate problem as a solution.</p>
<p>The emissions come from the infrastructure associated with AI, such as building and running the data centres that handle the large amounts of information required to sustain these systems.</p>
<p>But different technological approaches to how we build AI systems could help reduce its carbon footprint. Two technologies in particular hold promise for doing this: <a href="https://research.ibm.com/publications/spiking-neural-networks-enable-two-dimensional-neurons-and-unsupervised-multi-timescale-learning">spiking neural networks</a> and lifelong learning.</p>
<p>The lifetime of an AI system can be split into two phases: training and inference. During training, a relevant dataset is used to build and tune – improve – the system. In inference, the trained system generates predictions on previously unseen data.</p>
<p>For example, training an AI that’s to be used in self-driving cars would require a dataset of many different driving scenarios and decisions taken by human drivers.</p>
<p>After the training phase, the AI system will predict effective manoeuvres for a self-driving car. <a href="https://www.ibm.com/topics/neural-networks">Artificial neural networks (ANN)</a>, are an underlying technology used in most current AI systems. </p>
<p>They have many different elements to them, called parameters, whose values are adjusted during the training phase of the AI system. These parameters can run to more than 100 billion in total.</p>
<p>While large numbers of parameters improve the capabilities of ANNs, they also make training and inference resource-intensive processes. To put things in perspective, training GPT-3 (the precursor AI system to the current ChatGPT) generated 502 metric tonnes of carbon, which is equivalent to driving 112 petrol powered cars for a year. </p>
<p>GPT-3 further emits <a href="https://semiengineering.com/ai-power-consumption-exploding/">8.4 tonnes of CO₂ annually</a> due to inference. Since the AI boom started in the early 2010s, the energy requirements of AI systems known as large language models (LLMs) – the type of technology that’s behind ChatGPT – have gone up <a href="https://openai.com/research/ai-and-compute">by a factor of 300,000</a>. </p>
<p>With the increasing ubiquity and complexity of AI models, this trend is going to continue, potentially making AI a significant contributor of CO₂ emissions. In fact, our current estimates <a href="https://www.technologyreview.com/2023/12/05/1084417/ais-carbon-footprint-is-bigger-than-you-think/">could be lower than AI’s actual carbon footprint</a> due to a lack of standard and accurate techniques for measuring AI-related emissions.</p>
<figure class="align-center ">
<img alt="Chimneys at a power station." src="https://images.theconversation.com/files/576022/original/file-20240215-16-uqga7a.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=754&fit=clip" srcset="https://images.theconversation.com/files/576022/original/file-20240215-16-uqga7a.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=600&h=400&fit=crop&dpr=1 600w, https://images.theconversation.com/files/576022/original/file-20240215-16-uqga7a.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=600&h=400&fit=crop&dpr=2 1200w, https://images.theconversation.com/files/576022/original/file-20240215-16-uqga7a.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=600&h=400&fit=crop&dpr=3 1800w, https://images.theconversation.com/files/576022/original/file-20240215-16-uqga7a.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=754&h=503&fit=crop&dpr=1 754w, https://images.theconversation.com/files/576022/original/file-20240215-16-uqga7a.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=754&h=503&fit=crop&dpr=2 1508w, https://images.theconversation.com/files/576022/original/file-20240215-16-uqga7a.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=754&h=503&fit=crop&dpr=3 2262w" sizes="(min-width: 1466px) 754px, (max-width: 599px) 100vw, (min-width: 600px) 600px, 237px">
<figcaption>
<span class="caption"></span>
<span class="attribution"><a class="source" href="https://www.shutterstock.com/image-photo/aerial-view-high-smoke-stack-emission-1871428867">Leonid Sorokin / Shutterstock</a></span>
</figcaption>
</figure>
<h2>Spiking neural networks</h2>
<p>The previously mentioned new technologies, spiking neural networks (SNNs) and lifelong learning (L2), have the potential to lower AI’s ever-increasing carbon footprint, with SNNs acting as an energy-efficient alternative to ANNs. </p>
<p>ANNs work by processing and learning patterns from data, enabling them to make predictions. They work with decimal numbers. To make accurate calculations, especially when multiplying numbers with decimal points together, the computer needs to be very precise. It is because of these decimal numbers that ANNs require lots of computing power, memory and time. </p>
<p>This means ANNs become more energy-intensive as the networks get larger and more complex. Both ANNs and SNNs are inspired by the brain, which contains billions of neurons (nerve cells) connected to each other via synapses. </p>
<p>Like the brain, ANNs and SNNs also have components which researchers call neurons, although these are artificial, not biological ones. The key difference between the two types of neural networks is in the way individual neurons transmit information to each other. </p>
<p>Neurons in the human brain communicate with each other by transmitting intermittent electrical signals called spikes. The spikes themselves do not contain information. Instead, the information lies in the timing of these spikes. This binary, all-or-none characteristic of spikes (usually represented as 0 or 1) implies that neurons are active when they spike and inactive otherwise. </p>
<p>This is one of the reasons for <a href="https://www.frontiersin.org/journals/neuroscience/articles/10.3389/fnins.2019.00666/full">energy efficient processing in the brain</a>.</p>
<p>Just as Morse code uses specific sequences of dots and dashes to convey messages, SNNs use patterns or timings of spikes to process and transmit information. So, while the artificial neurons in ANNs are always active, SNNs consume energy only when a spike occurs. </p>
<p>Otherwise, they have closer to zero energy requirements. SNNs <a href="https://arxiv.org/abs/1903.06530">can be up to 280 times</a> more energy efficient than ANNs.</p>
<p>My colleagues and I are <a href="https://www.biorxiv.org/content/10.1101/2023.04.03.535317v1.abstract">developing learning algorithms for SNNs</a> that may bring them even closer to the energy efficiency exhibited by the brain. The lower computational requirements also imply that SNNs <a href="https://ieeexplore.ieee.org/abstract/document/10191334">might be able to make decisions more quickly</a>. </p>
<p>These properties render SNNs useful for broad range of applications, including space exploration, <a href="https://arxiv.org/abs/2305.16748">defence</a> and self-driving cars because of the limited energy sources available in these scenarios.</p>
<h1>Lifelong learning</h1>
<p>L2 is another strategy for reducing the overall energy requirements of ANNs over the course of their lifetime that we are also working on. </p>
<p>Training ANNs sequentially (where the systems learn from sequences of data) on new problems causes them to forget <a href="https://www.sciencedirect.com/science/article/abs/pii/S0079742108605368?via%3Dihub">their previous knowledge</a> while learning new tasks. ANNs require retraining from scratch when their operating environment changes, further increasing AI-related emissions. </p>
<p>L2 is a collection of algorithms that enable AI models to be trained sequentially on multiple tasks with little or no forgetting. L2 enables models to <a href="https://arxiv.org/abs/2305.16748">learn throughout their lifetime</a> by building on their existing knowledge without having to retrain them from scratch.</p>
<p>The field of AI is growing fast and other potential advancements are emerging that can mitigate the energy demands of this technology. For instance, building smaller AI models that exhibit the same predictive capabilities as that of a larger model. </p>
<p>Advances in quantum computing – a different approach to building computers that harnesses phenomena from the world of quantum physics – would also enable faster training and inference using ANNs and SNNs. The superior computing capabilities offered by quantum computing could allow us to find energy-efficient solutions for AI at a much larger scale.</p>
<p>The climate change challenge requires that we try to find solutions for rapidly advancing areas such as AI before their carbon footprint becomes too large.</p><img src="https://counter.theconversation.com/content/223488/count.gif" alt="The Conversation" width="1" height="1" />
<p class="fine-print"><em><span>Shirin Dora does not work for, consult, own shares in or receive funding from any company or organisation that would benefit from this article, and has disclosed no relevant affiliations beyond their academic appointment.</span></em></p>Technological approaches could help reduce the carbon impact of artificial intelligence systems.Shirin Dora, Lecturer, Computer Science, Loughborough UniversityLicensed as Creative Commons – attribution, no derivatives.tag:theconversation.com,2011:article/2167302023-11-01T10:10:34Z2023-11-01T10:10:34ZWe built a ‘brain’ from tiny silver wires. It learns in real time, more efficiently than computer-based AI<figure><img src="https://images.theconversation.com/files/556971/original/file-20231031-15-3i8f9x.jpg?ixlib=rb-1.1.0&rect=0%2C7%2C2556%2C1908&q=45&auto=format&w=496&fit=clip" /><figcaption><span class="caption">
</span> <span class="attribution"><a class="source" href="https://doi.org/10.1038/s41467-023-42470-5">Zhu et al. / Nature Communications</a></span></figcaption></figure><p>The world is infatuated with artificial intelligence (AI), and for good reason. AI systems can process vast quantities of data in a seemingly superhuman way.</p>
<p>However, current AI systems rely on computers running complex algorithms based on <a href="https://arxiv.org/abs/2212.11279">artificial neural networks</a>. These use <a href="https://www.numenta.com/blog/2022/05/24/ai-is-harming-our-planet/">huge amounts of energy</a>, and use even more energy if you are trying to work with data that changes in real time.</p>
<p>We are working on a completely new approach to “machine intelligence”. Instead of using artificial neural network software, we have developed a <em>physical</em> neural network in hardware that operates much more efficiently.</p>
<p>Our neural networks, made from silver nanowires, can learn on the fly to recognise handwritten numbers and memorise strings of digits. Our results are published in <a href="https://doi.org/10.1038/s41467-023-42470-5">a new paper</a> in Nature Communications, conducted with colleagues from the University of Sydney and the University of California, Los Angeles.</p>
<h2>A random network of tiny wires</h2>
<p>Using nanotechnology, we made networks of silver nanowires about one thousandth the width of a human hair. These nanowires naturally form a random network, much like the pile of sticks in a game of pick-up sticks. </p>
<p>The nanowires’ network structure looks a lot like the network of neurons in our brains. Our research is part of a field called <a href="https://www.nature.com/articles/s41928-021-00646-1">neuromorphic computing</a>, which aims to emulate the brain-like functionality of neurons and synapses in hardware. </p>
<figure class="align-center zoomable">
<a href="https://images.theconversation.com/files/556993/original/file-20231101-27-46gvu1.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=1000&fit=clip"><img alt="A microscope photo showing a messy web of thin grey lines against a black background." src="https://images.theconversation.com/files/556993/original/file-20231101-27-46gvu1.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=754&fit=clip" srcset="https://images.theconversation.com/files/556993/original/file-20231101-27-46gvu1.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=600&h=450&fit=crop&dpr=1 600w, https://images.theconversation.com/files/556993/original/file-20231101-27-46gvu1.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=600&h=450&fit=crop&dpr=2 1200w, https://images.theconversation.com/files/556993/original/file-20231101-27-46gvu1.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=600&h=450&fit=crop&dpr=3 1800w, https://images.theconversation.com/files/556993/original/file-20231101-27-46gvu1.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=754&h=566&fit=crop&dpr=1 754w, https://images.theconversation.com/files/556993/original/file-20231101-27-46gvu1.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=754&h=566&fit=crop&dpr=2 1508w, https://images.theconversation.com/files/556993/original/file-20231101-27-46gvu1.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=754&h=566&fit=crop&dpr=3 2262w" sizes="(min-width: 1466px) 754px, (max-width: 599px) 100vw, (min-width: 600px) 600px, 237px"></a>
<figcaption>
<span class="caption">Each nanowire is around one thousandth the width of a human hair, and together they form a random network that behaves much like the web of neurons in our brains.</span>
<span class="attribution"><a class="source" href="https://doi.org/10.1038/s41467-023-42470-5">Zhu et al. / Nature Communications</a></span>
</figcaption>
</figure>
<p>Our nanowire networks display brain-like behaviours in response to electrical signals. External electrical signals cause changes in how electricity is transmitted at the points where nanowires intersect, which is similar to how biological <a href="https://qbi.uq.edu.au/brain-basics/brain/brain-physiology/action-potentials-and-synapses">synapses</a> work. </p>
<p>There can be tens of thousands of synapse-like intersections in a typical nanowire network, which means the network can efficiently process and transmit information carried by electrical signals.</p>
<h2>Learning and adapting in real time</h2>
<p>In our study, we show that because nanowire networks can respond to signals that change in time, they can be used for <a href="https://medium.com/value-stream-design/online-machine-learning-515556ff72c5">online machine learning</a>. </p>
<p>In conventional machine learning, data is fed into the system and processed in <a href="https://towardsdatascience.com/batch-mini-batch-stochastic-gradient-descent-7a62ecba642a">batches</a>. In the online learning approach, we can introduce data to the system as a continuous stream in time. </p>
<p>With each new piece of data, the system learns and adapts in real time. It demonstrates “on the fly” learning, which we humans are good at but current AI systems are not. </p>
<hr>
<p>
<em>
<strong>
Read more:
<a href="https://theconversation.com/networks-of-silver-nanowires-seem-to-learn-and-remember-much-like-our-brains-204115">Networks of silver nanowires seem to learn and remember, much like our brains</a>
</strong>
</em>
</p>
<hr>
<p>The online learning approach enabled by our nanowire network is more efficient than conventional batch-based learning in AI applications. </p>
<p>In batch learning, a significant amount of memory is needed to process large datasets, and the system often needs to go through the same data multiple times to learn. This not only demands high computational resources but also consumes more energy overall. </p>
<p>Our online approach requires less memory as data is processed continuously. Moreover, our network learns from each data sample only once, significantly reducing energy use and making the process highly efficient.</p>
<h2>Recognising and remembering numbers</h2>
<p>We tested the nanowire network with a benchmark image recognition task using the <a href="https://paperswithcode.com/dataset/mnist">MNIST dataset</a> of handwritten digits. </p>
<p>The greyscale pixel values in the images were converted to electrical signals and fed into the network. After each digit sample, the network learned and refined its ability to recognise the patterns, displaying real-time learning.</p>
<figure class="align-center zoomable">
<a href="https://images.theconversation.com/files/557006/original/file-20231101-25-pghp65.png?ixlib=rb-1.1.0&q=45&auto=format&w=1000&fit=clip"><img alt="A grid of handwritten digits" src="https://images.theconversation.com/files/557006/original/file-20231101-25-pghp65.png?ixlib=rb-1.1.0&q=45&auto=format&w=754&fit=clip" srcset="https://images.theconversation.com/files/557006/original/file-20231101-25-pghp65.png?ixlib=rb-1.1.0&q=45&auto=format&w=600&h=352&fit=crop&dpr=1 600w, https://images.theconversation.com/files/557006/original/file-20231101-25-pghp65.png?ixlib=rb-1.1.0&q=30&auto=format&w=600&h=352&fit=crop&dpr=2 1200w, https://images.theconversation.com/files/557006/original/file-20231101-25-pghp65.png?ixlib=rb-1.1.0&q=15&auto=format&w=600&h=352&fit=crop&dpr=3 1800w, https://images.theconversation.com/files/557006/original/file-20231101-25-pghp65.png?ixlib=rb-1.1.0&q=45&auto=format&w=754&h=443&fit=crop&dpr=1 754w, https://images.theconversation.com/files/557006/original/file-20231101-25-pghp65.png?ixlib=rb-1.1.0&q=30&auto=format&w=754&h=443&fit=crop&dpr=2 1508w, https://images.theconversation.com/files/557006/original/file-20231101-25-pghp65.png?ixlib=rb-1.1.0&q=15&auto=format&w=754&h=443&fit=crop&dpr=3 2262w" sizes="(min-width: 1466px) 754px, (max-width: 599px) 100vw, (min-width: 600px) 600px, 237px"></a>
<figcaption>
<span class="caption">The nanowire network learned to recognise handwritten numbers, a common benchmark for machine learning systems.</span>
<span class="attribution"><a class="source" href="https://en.wikipedia.org/wiki/MNIST_database#/media/File:MnistExamplesModified.png">NIST / Wikimedia</a>, <a class="license" href="http://creativecommons.org/licenses/by-sa/4.0/">CC BY-SA</a></span>
</figcaption>
</figure>
<p>Using the same learning method, we also tested the nanowire network with a memory task involving patterns of digits, much like the process of remembering a phone number. The network demonstrated an ability to remember previous digits in the pattern. </p>
<p>Overall, these tasks demonstrate the network’s potential for emulating brain-like learning and memory. Our work has so far only scratched the surface of what neuromorphic nanowire networks can do.</p><img src="https://counter.theconversation.com/content/216730/count.gif" alt="The Conversation" width="1" height="1" />
<p class="fine-print"><em><span>Zdenka Kuncic owns shares in Emergentia, Inc., and acknowledges support from the Australian-American Fulbright Commission.</span></em></p><p class="fine-print"><em><span>Ruomin Zhu receives the PREA scholarship from the University of Sydney. </span></em></p>A tangle of silver nanowires may pave the way to low-energy real-time machine learning.Zdenka Kuncic, Professor of Physics, University of SydneyRuomin Zhu, PhD student, University of SydneyLicensed as Creative Commons – attribution, no derivatives.tag:theconversation.com,2011:article/1928002022-10-19T18:37:54Z2022-10-19T18:37:54ZA new type of material called a mechanical neural network can learn and change its physical properties to create adaptable, strong structures<figure><img src="https://images.theconversation.com/files/490679/original/file-20221019-12170-qt1idp.JPG?ixlib=rb-1.1.0&rect=48%2C78%2C3977%2C2939&q=45&auto=format&w=496&fit=clip" /><figcaption><span class="caption">This connection of springs is a new type of material that can change shape and learn new properties.</span> <span class="attribution"><span class="source">Jonathan Hopkins</span>, <a class="license" href="http://creativecommons.org/licenses/by-nd/4.0/">CC BY-ND</a></span></figcaption></figure><p><em>The <a href="https://theconversation.com/us/topics/research-brief-83231">Research Brief</a> is a short take about interesting academic work.</em></p>
<h2>The big idea</h2>
<p>A new type of material can learn and improve its ability to deal with unexpected forces thanks to a unique lattice structure with connections of variable stiffness, as <a href="https://doi.org/10.1126/scirobotics.abq7278">described in a new paper</a> by my colleagues and me. </p>
<figure class="align-right zoomable">
<a href="https://images.theconversation.com/files/490682/original/file-20221019-23-rnqscu.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=1000&fit=clip"><img alt="A hand holding a small, complex cube of plastic." src="https://images.theconversation.com/files/490682/original/file-20221019-23-rnqscu.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=237&fit=clip" srcset="https://images.theconversation.com/files/490682/original/file-20221019-23-rnqscu.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=600&h=600&fit=crop&dpr=1 600w, https://images.theconversation.com/files/490682/original/file-20221019-23-rnqscu.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=600&h=600&fit=crop&dpr=2 1200w, https://images.theconversation.com/files/490682/original/file-20221019-23-rnqscu.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=600&h=600&fit=crop&dpr=3 1800w, https://images.theconversation.com/files/490682/original/file-20221019-23-rnqscu.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=754&h=754&fit=crop&dpr=1 754w, https://images.theconversation.com/files/490682/original/file-20221019-23-rnqscu.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=754&h=754&fit=crop&dpr=2 1508w, https://images.theconversation.com/files/490682/original/file-20221019-23-rnqscu.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=754&h=754&fit=crop&dpr=3 2262w" sizes="(min-width: 1466px) 754px, (max-width: 599px) 100vw, (min-width: 600px) 600px, 237px"></a>
<figcaption>
<span class="caption">Architected materials – like this 3D lattice – get their properties not from what they are made out of, but from their structure.</span>
<span class="attribution"><span class="source">Ryan Lee</span>, <a class="license" href="http://creativecommons.org/licenses/by-nd/4.0/">CC BY-ND</a></span>
</figcaption>
</figure>
<p>The new material is a type of architected material, which gets its properties mainly from the geometry and specific traits of its design rather than what it is made out of. Take hook-and-loop fabric closures like Velcro, for example. It doesn’t matter whether it is made from cotton, plastic or any other substance. As long as one side is a fabric with stiff hooks and the other side has fluffy loops, the material will have the sticky properties of Velcro.</p>
<p>My colleagues and I based our new material’s architecture on that of an artificial neural network – layers of interconnected nodes that can <a href="https://doi.org/10.1109/ACCESS.2019.2945545">learn to do tasks</a> by changing how much importance, or weight, they place on each connection. We hypothesized that a mechanical lattice with physical nodes could be trained to take on certain mechanical properties by adjusting each connection’s rigidity. </p>
<p>To find out if a mechanical lattice would be able to adopt and maintain new properties – like taking on a new shape or changing directional strength – we started off by building a computer model. We then selected a desired shape for the material as well as input forces and had a computer algorithm tune the tensions of the connections so that the input forces would produce the desired shape. We did this training on 200 different lattice structures and found that a triangular lattice was best at achieving all of the shapes we tested. </p>
<p>Once the many connections are tuned to achieve a set of tasks, the material will continue to react in the desired way. The training is – in a sense – remembered in the structure of the material itself.</p>
<p>We then built a physical prototype lattice with adjustable electromechanical springs arranged in a triangular lattice. The prototype is made of 6-inch connections and is about 2 feet long by 1½ feet wide. And it worked. When the lattice and algorithm worked together, the material was able to learn and change shape in particular ways when subjected to different forces. We call this new material a mechanical neural network.</p>
<figure class="align-center zoomable">
<a href="https://images.theconversation.com/files/490683/original/file-20221019-14-emmwwr.JPG?ixlib=rb-1.1.0&q=45&auto=format&w=1000&fit=clip"><img alt="A photo of hydraulic springs arranged in a triangular lattice" src="https://images.theconversation.com/files/490683/original/file-20221019-14-emmwwr.JPG?ixlib=rb-1.1.0&q=45&auto=format&w=754&fit=clip" srcset="https://images.theconversation.com/files/490683/original/file-20221019-14-emmwwr.JPG?ixlib=rb-1.1.0&q=45&auto=format&w=600&h=450&fit=crop&dpr=1 600w, https://images.theconversation.com/files/490683/original/file-20221019-14-emmwwr.JPG?ixlib=rb-1.1.0&q=30&auto=format&w=600&h=450&fit=crop&dpr=2 1200w, https://images.theconversation.com/files/490683/original/file-20221019-14-emmwwr.JPG?ixlib=rb-1.1.0&q=15&auto=format&w=600&h=450&fit=crop&dpr=3 1800w, https://images.theconversation.com/files/490683/original/file-20221019-14-emmwwr.JPG?ixlib=rb-1.1.0&q=45&auto=format&w=754&h=566&fit=crop&dpr=1 754w, https://images.theconversation.com/files/490683/original/file-20221019-14-emmwwr.JPG?ixlib=rb-1.1.0&q=30&auto=format&w=754&h=566&fit=crop&dpr=2 1508w, https://images.theconversation.com/files/490683/original/file-20221019-14-emmwwr.JPG?ixlib=rb-1.1.0&q=15&auto=format&w=754&h=566&fit=crop&dpr=3 2262w" sizes="(min-width: 1466px) 754px, (max-width: 599px) 100vw, (min-width: 600px) 600px, 237px"></a>
<figcaption>
<span class="caption">The prototype is 2D, but a 3D version of this material could have many uses.</span>
<span class="attribution"><span class="source">Jonathan Hopkins</span>, <a class="license" href="http://creativecommons.org/licenses/by-nd/4.0/">CC BY-ND</a></span>
</figcaption>
</figure>
<h2>Why it matters</h2>
<p>Besides some <a href="https://doi.org/10.1007/BF00436764">living tissues</a>, very few materials can learn to be better at dealing with unanticipated loads. Imagine a plane wing that suddenly catches a gust of wind and is forced in an unanticipated direction. The wing can’t change its design to be stronger in that direction.</p>
<p>The prototype lattice material we designed can adapt to changing or unknown conditions. In a wing, for example, these changes could be the accumulation of internal damage, changes in how the wing is attached to a craft or fluctuating external loads. Every time a wing made out of a mechanical neural network experienced one of these scenarios, it could strengthen and soften its connections to maintain desired attributes like directional strength. Over time, through successive adjustments made by the algorithm, the wing adopts and maintains new properties, adding each behavior to the rest as a sort of muscle memory.</p>
<p>This type of material could have far reaching applications for the longevity and efficiency of built structures. Not only could a wing made of a mechanical neural network material be stronger, it could also be trained to morph into shapes that maximize fuel efficiency in response to changing conditions around it.</p>
<h2>What’s still not known</h2>
<p>So far, our team has worked only with 2D lattices. But using computer modeling, we predict that 3D lattices would have a much larger capacity for learning and adaptation. This increase is due to the fact that a 3D structure could have tens of times more connections, or springs, that don’t intersect with one another. However, the mechanisms we used in our first model are far too complex to support in a large 3D structure. </p>
<h2>What’s next</h2>
<p>The material my colleagues and I created is a proof of concept and shows the potential of mechanical neural networks. But to bring this idea into the real world will require figuring out how to make the individual pieces smaller and with precise properties of flex and tension.</p>
<p>We hope new research in the <a href="https://doi.org/10.1039/C8MH01100A">manufacturing of materials at the micron scale</a>, as well as work on <a href="https://doi.org/10.1016/j.eml.2020.101120">new materials with adjustable stiffness</a>, will lead to advances that make powerful smart mechanical neural networks with micron-scale elements and dense 3D connections a ubiquitous reality in the near future.</p><img src="https://counter.theconversation.com/content/192800/count.gif" alt="The Conversation" width="1" height="1" />
<p class="fine-print"><em><span>Ryan Lee has received funding from the Air Force Office of Science Research . </span></em></p>Computer-based neural networks can learn to do tasks. A new type of material, called a mechanical neural network, applies similar ideas to a physical structure.Ryan H. Lee, PhD Student in Mechanical and Aerospace Engineering, University of California, Los AngelesLicensed as Creative Commons – attribution, no derivatives.tag:theconversation.com,2011:article/1839932022-06-06T04:06:53Z2022-06-06T04:06:53ZWe’re told AI neural networks ‘learn’ the way humans do. A neuroscientist explains why that’s not the case<figure><img src="https://images.theconversation.com/files/467087/original/file-20220606-14-tpqupv.jpeg?ixlib=rb-1.1.0&rect=0%2C0%2C6261%2C3732&q=45&auto=format&w=496&fit=clip" /><figcaption><span class="caption">
</span> <span class="attribution"><span class="source">Shutterstock</span></span></figcaption></figure><p>Recently developed artificial intelligence (AI) models are capable of many impressive feats, including recognising images and producing human-like language. But just because AI can perform human-like behaviours doesn’t mean it can think or understand like humans.</p>
<p>As a researcher studying how humans understand and reason about the world, I think it’s important to emphasise the way AI systems “think” and learn is fundamentally different to how humans do – and we have a long way to go before AI can truly think like us.</p>
<hr>
<p>
<em>
<strong>
Read more:
<a href="https://theconversation.com/robots-are-creating-images-and-telling-jokes-5-things-to-know-about-foundation-models-and-the-next-generation-of-ai-181150">Robots are creating images and telling jokes. 5 things to know about foundation models and the next generation of AI</a>
</strong>
</em>
</p>
<hr>
<h2>A widespread misconception</h2>
<p>Developments in AI have produced systems that can perform very human-like behaviours. The language model <a href="https://www.twilio.com/blog/ultimate-guide-openai-gpt-3-language-model">GPT-3</a> can produce text that’s often indistinguishable from human speech. Another model, <a href="https://ai.googleblog.com/2022/04/pathways-language-model-palm-scaling-to.html">PaLM</a>, can produce explanations for jokes it has never <a href="https://www.cnet.com/tech/services-and-software/no-joke-googles-ai-is-smart-enough-to-understand-your-humor/">seen before</a>. </p>
<p>Most recently, a general-purpose AI known as Gato has been developed which can <a href="https://www.zdnet.com/article/deepminds-gato-is-mediocre-so-why-did-they-build-it/">perform hundreds of tasks</a>, including captioning images, answering questions, playing Atari video games, and even controlling a robot arm to stack blocks. And DALL-E is a system which has been trained to produce modified images and artwork from a text description.</p>
<p><div data-react-class="Tweet" data-react-props="{"tweetId":"1531507253659914240"}"></div></p>
<p>These breakthroughs have led to some bold claims about the capability of such AI, and what it can tell us about human intelligence. </p>
<p>For example Nando de Freitas, a researcher at Google’s AI company DeepMind, argues scaling up existing models will be enough to produce human-level artificial intelligence. Others have <a href="https://medium.com/@blaisea/do-large-language-models-understand-us-6f881d6d8e75">echoed</a> this view.</p>
<p><div data-react-class="Tweet" data-react-props="{"tweetId":"1525397036325019649"}"></div></p>
<p>In all the excitement, it’s easy to assume human-like behaviour means human-like understanding. But there are several key differences between how AI and humans think and learn. </p>
<h2>Neural nets vs the human brain</h2>
<p>Most recent AI is built from <a href="https://theconversation.com/what-is-a-neural-network-a-computer-scientist-explains-151897">artificial neural networks</a>, or “neural nets” for short. The term “neural” is used because these networks are inspired by the human brain, in which billions of cells called neurons form complex webs of connections with one another, processing information as they fire signals back and forth. </p>
<p>Neural nets are a highly simplified version of the biology. A real neuron is replaced with a simple node, and the strength of the connection between nodes is represented by a single number called a “weight”.</p>
<p>With enough connected nodes stacked into enough layers, neural nets can be trained to recognise patterns and even “<a href="https://medium.com/deep-learning-demystified/generalization-in-neural-networks-7765ee42ac23">generalise</a>” to stimuli that are similar (but not identical) to what they’ve seen before. Simply, generalisation refers to an AI system’s ability to take what it has learnt from certain data and apply it to new data.</p>
<p>Being able to identify features, recognise patterns, and generalise from results lies at the heart of the success of neural nets – and mimics techniques humans use for such tasks. Yet there are important differences.</p>
<p>Neural nets are typically trained by “<a href="https://www.ibm.com/cloud/learn/supervised-learning">supervised learning</a>”. So they’re presented with many examples of an input and the desired output, and then gradually the connection weights are adjusted until the network “learns” to produce the desired output. </p>
<p>To learn a language task, a neural net may be presented with a sentence one word at a time, and will slowly learns to predict the next word in the sequence. </p>
<p>This is very different from how humans typically learn. Most human learning is “unsupervised”, which means we’re not explicitly told what the “right” response is for a given stimulus. We have to work this out ourselves. </p>
<p>For instance, children aren’t given instructions on how to speak, but learn this through a <a href="https://www.abc.net.au/education/how-babies-learn-to-talk/13757672">complex process</a> of exposure to adult speech, imitation, and feedback.</p>
<figure class="align-center ">
<img alt="A toddler tries to walk outdoors, with an adult guiding it by both hands" src="https://images.theconversation.com/files/467090/original/file-20220606-22-7fwow4.jpeg?ixlib=rb-1.1.0&q=45&auto=format&w=754&fit=clip" srcset="https://images.theconversation.com/files/467090/original/file-20220606-22-7fwow4.jpeg?ixlib=rb-1.1.0&q=45&auto=format&w=600&h=338&fit=crop&dpr=1 600w, https://images.theconversation.com/files/467090/original/file-20220606-22-7fwow4.jpeg?ixlib=rb-1.1.0&q=30&auto=format&w=600&h=338&fit=crop&dpr=2 1200w, https://images.theconversation.com/files/467090/original/file-20220606-22-7fwow4.jpeg?ixlib=rb-1.1.0&q=15&auto=format&w=600&h=338&fit=crop&dpr=3 1800w, https://images.theconversation.com/files/467090/original/file-20220606-22-7fwow4.jpeg?ixlib=rb-1.1.0&q=45&auto=format&w=754&h=424&fit=crop&dpr=1 754w, https://images.theconversation.com/files/467090/original/file-20220606-22-7fwow4.jpeg?ixlib=rb-1.1.0&q=30&auto=format&w=754&h=424&fit=crop&dpr=2 1508w, https://images.theconversation.com/files/467090/original/file-20220606-22-7fwow4.jpeg?ixlib=rb-1.1.0&q=15&auto=format&w=754&h=424&fit=crop&dpr=3 2262w" sizes="(min-width: 1466px) 754px, (max-width: 599px) 100vw, (min-width: 600px) 600px, 237px">
<figcaption>
<span class="caption">Childrens’ learning is assisted by adults, but they’re not fed massive datasets the way AI systems are.</span>
<span class="attribution"><span class="source">Shutterstock</span></span>
</figcaption>
</figure>
<p>Another difference is the sheer scale of data used to train AI. The GPT-3 model was trained on <a href="https://arxiv.org/pdf/2005.14165.pdf">400 billion words</a>, mostly taken from the internet. At a rate of 150 words per minute, it would take a human nearly 4,000 years to read this much text. </p>
<p>Such calculations show humans can’t possibly learn the same way AI does. We have to make more efficient use of smaller amounts of data.</p>
<h2>Neural nets can learn in ways we can’t</h2>
<p>An even more fundamental difference concerns the way neural nets learn. In order to match up a stimulus with a desired response, neural nets use an algorithm called “backpropagation” to pass errors backward through the network, allowing the weights to be adjusted in just the right way. </p>
<p>However, it’s widely recognised by neuroscientists that <a href="https://www.nature.com/articles/s41583-020-0277-3/">backpropagation can’t be implemented</a> in the brain, as it would require <a href="https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0266102">external signals</a> that just don’t exist.</p>
<p>Some researchers have proposed <a href="https://www.sciencedirect.com/science/article/pii/S1364661319300129">variations</a> of backpropagation could be used by the brain, but so far there is no evidence human brains can use such learning methods.</p>
<p>Instead, humans learn by making <a href="https://link.springer.com/article/10.1007/s12559-014-9307-7">structured mental concepts</a>, in which many different properties and associations are linked together. For instance, our concept of “banana” includes its shape, the colour yellow, knowledge of it being a fruit, how to hold it, and so forth. </p>
<p>As far as we know, AI systems do not form conceptual knowledge like this. They rely entirely on extracting complex statistical associations from their training data, and then applying these to similar contexts. </p>
<p>Efforts are underway to build AI that <a href="https://www.techrepublic.com/article/multimodal-learning-the-future-of-artificial-intelligence/">combines different types of input</a> (such as images and text) – but it remains to be seen if this will be sufficient for these models to learn the same types of rich mental representations humans use to understand the world.</p>
<p>There’s still much we don’t know about how humans learn, understand and reason. However, what we do know indicates humans perform these tasks very differently to AI systems. </p>
<p>As such, <a href="https://www.zdnet.com/article/resisting-the-urge-to-be-impressed-and-knowing-what-we-are-talking-about-when-we-talk-about-ai/">many researchers believe</a> we’ll need new approaches, and more fundamental insight into how the human brain works, before we can build machines that truly think and learn like humans.</p><img src="https://counter.theconversation.com/content/183993/count.gif" alt="The Conversation" width="1" height="1" />
<p class="fine-print"><em><span>James Fodor is a PhD candidate at the Brain, Mind & Markets Laboratory, Department of Finance, Faculty of Business and Economics, University of Melbourne.</span></em></p>We’ve seen AI systems writing texts that are indistinguishable from human texts. Some are even rendering impressive 3D artworks from short text inputs. But it doesn’t mean they can ‘think’ like us.James Fodor, PhD Candidate in Cognitive Neuroscience, The University of MelbourneLicensed as Creative Commons – attribution, no derivatives.tag:theconversation.com,2011:article/1810402022-04-29T04:10:36Z2022-04-29T04:10:36ZHoneybees join humans as the only known animals that can tell the difference between odd and even numbers<figure><img src="https://images.theconversation.com/files/460466/original/file-20220429-23745-noj25i.jpeg?ixlib=rb-1.1.0&rect=215%2C1012%2C3096%2C1571&q=45&auto=format&w=496&fit=clip" /><figcaption><span class="caption">
</span> <span class="attribution"><span class="source">Shutterstock</span></span></figcaption></figure><p>“Two, four, six, eight; bog in, don’t wait”. </p>
<p>As children, we learn numbers can either be even or odd. And there are many ways to categorise numbers as even or odd.</p>
<p>We may memorise the rule that numbers ending in 1, 3, 5, 7, or 9 are odd while numbers ending in 0, 2, 4, 6, or 8 are even. Or we may divide a number by 2 – where any whole number outcome means the number is even, otherwise it must be odd. </p>
<p>Similarly, when dealing with real-world objects we can use pairing. If we have an unpaired element left over, that means the number of objects was odd.</p>
<p>Until now odd and even categorisation, also called parity classification, had never been shown in non-human animals. In a new study, <a href="https://www.frontiersin.org/articles/10.3389/fevo.2022.805385/full">published today</a> in the journal Frontiers in Ecology and Evolution, we show honeybees can learn to do this.</p>
<h2>Why is parity categorisation special?</h2>
<p>Parity tasks (such as odd and even categorisation) are considered abstract and high-level <a href="https://psycnet.apa.org/record/1993-44067-001?doi=1">numerical concepts in humans</a>.</p>
<p>Interestingly, humans demonstrate accuracy, speed, language and spatial relationship biases when categorising numbers as odd or even. For example, we <a href="https://www.sciencedirect.com/science/article/pii/S0010027799000487">tend to respond faster</a> to even numbers with actions performed by our right hand, and to odd numbers with actions performed by our left hand. </p>
<p>We are also faster, and more accurate, when categorising numbers as even compared to odd. And research has found children typically associate the word <a href="https://www.sciencedirect.com/science/article/abs/pii/S0022096599925183">“even” with “right” and “odd” with “left”</a>.</p>
<p>These studies suggest humans may have learnt biases and/or innate biases regarding odd and even numbers, which may have arisen either through evolution, cultural transmission, or a combination of both. </p>
<p>It isn’t obvious why parity might be important beyond its use in mathematics, so the origins of these biases remain unclear. Understanding if and how other animals can recognise (or can learn to recognise) odd and even numbers could tell us more about our own history with parity.</p>
<h2>Training bees to learn odd and even</h2>
<p>Studies have shown honeybees can learn to <a href="https://theconversation.com/bees-join-an-elite-group-of-species-that-understands-the-concept-of-zero-as-a-number-97316">order quantities</a>, perform simple addition and subtraction, <a href="https://theconversation.com/we-taught-bees-a-simple-number-language-and-they-got-it-117816">match symbols with quantities</a> and <a href="https://www.sciencedirect.com/science/article/pii/S2589004220303072">relate size and number concepts</a>. </p>
<hr>
<p>
<em>
<strong>
Read more:
<a href="https://theconversation.com/can-bees-do-maths-yes-new-research-shows-they-can-add-and-subtract-108074">Can bees do maths? Yes – new research shows they can add and subtract</a>
</strong>
</em>
</p>
<hr>
<p>To teach bees a parity task, we separated individuals into two groups. One was trained to associate even numbers with sugar water and odd numbers with a bitter-tasting liquid (quinine). The other group was trained to associate odd numbers with sugar water, and even numbers with quinine.</p>
<figure class="align-center ">
<img alt="Image shows a schematic of a honeybee being shown an array of odd vs. even quantities on a circular screen in three different trials." src="https://images.theconversation.com/files/457822/original/file-20220413-12-dpkge8.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=754&fit=clip" srcset="https://images.theconversation.com/files/457822/original/file-20220413-12-dpkge8.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=600&h=352&fit=crop&dpr=1 600w, https://images.theconversation.com/files/457822/original/file-20220413-12-dpkge8.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=600&h=352&fit=crop&dpr=2 1200w, https://images.theconversation.com/files/457822/original/file-20220413-12-dpkge8.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=600&h=352&fit=crop&dpr=3 1800w, https://images.theconversation.com/files/457822/original/file-20220413-12-dpkge8.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=754&h=442&fit=crop&dpr=1 754w, https://images.theconversation.com/files/457822/original/file-20220413-12-dpkge8.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=754&h=442&fit=crop&dpr=2 1508w, https://images.theconversation.com/files/457822/original/file-20220413-12-dpkge8.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=754&h=442&fit=crop&dpr=3 2262w" sizes="(min-width: 1466px) 754px, (max-width: 599px) 100vw, (min-width: 600px) 600px, 237px">
<figcaption>
<span class="caption">Here we show a honeybee being trained to associate ‘even’ stimuli with a reward over 40 training choices.</span>
<span class="attribution"><span class="source">Scarlett Howard</span></span>
</figcaption>
</figure>
<p>We trained individual bees using comparisons of odd versus even numbers (with cards presenting 1-10 printed shapes) until they chose the correct answer with 80% accuracy.</p>
<p>Remarkably, the respective groups learnt at different rates. The bees trained to associate odd numbers with sugar water learnt quicker. Their learning bias towards odd numbers was the opposite of humans, who categorise even numbers more quickly.</p>
<figure class="align-right ">
<img alt="Honeybee standing on a grey plexiglass paltform ridge drinking a clear liquid (sugar water)." src="https://images.theconversation.com/files/457280/original/file-20220411-19-z8jc37.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=237&fit=clip" srcset="https://images.theconversation.com/files/457280/original/file-20220411-19-z8jc37.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=600&h=834&fit=crop&dpr=1 600w, https://images.theconversation.com/files/457280/original/file-20220411-19-z8jc37.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=600&h=834&fit=crop&dpr=2 1200w, https://images.theconversation.com/files/457280/original/file-20220411-19-z8jc37.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=600&h=834&fit=crop&dpr=3 1800w, https://images.theconversation.com/files/457280/original/file-20220411-19-z8jc37.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=754&h=1048&fit=crop&dpr=1 754w, https://images.theconversation.com/files/457280/original/file-20220411-19-z8jc37.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=754&h=1048&fit=crop&dpr=2 1508w, https://images.theconversation.com/files/457280/original/file-20220411-19-z8jc37.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=754&h=1048&fit=crop&dpr=3 2262w" sizes="(min-width: 1466px) 754px, (max-width: 599px) 100vw, (min-width: 600px) 600px, 237px">
<figcaption>
<span class="caption">Honeybees landed on a platform to drink sugar water during the experiment.</span>
<span class="attribution"><span class="source">Scarlett Howard</span></span>
</figcaption>
</figure>
<p>We then tested each bee on new numbers not shown during the training. Impressively, they categorised the new numbers of 11 or 12 elements as odd or even with an accuracy of about 70%. </p>
<p>Our results showed the miniature brains of honeybees were able to understand the concepts of odd and even. So a large and complex human brain <a href="https://www.nature.com/scitable/blog/brain-metrics/are_there_really_as_many/#:%7E:text=Approximately%2086%20billion%20neurons%20in,between%20200%20and%20400%20billion.">consisting of 86 billion neurons</a>, and a miniature insect brain <a href="https://rsv.org.au/one-two-bee/#:%7E:text=The%20brains%20of%20bees%20are,neural%20processing%20capabilities%20in%20comparison.">with about 960,000 neurons</a>, could both categorise numbers by parity.</p>
<p>Does this mean the parity task was less complex than we’d previously thought? To find the answer, we turned to bio-inspired technology.</p>
<figure>
<iframe width="440" height="260" src="https://www.youtube.com/embed/0dMRzGQKKLU?wmode=transparent&start=0" frameborder="0" allowfullscreen=""></iframe>
<figcaption><span class="caption">We trained honeybees to choose even numbers. In this video we see the bee inspect each card on the screen, before making a correct choice on the card presenting an even number of 12 shapes.</span></figcaption>
</figure>
<h2>Creating a simple artificial neural network</h2>
<p>Artificial neural networks were one of the first learning algorithms developed for machine learning. Inspired by biological neurons, these networks are scalable and can tackle complex recognition and classification tasks using <a href="https://link.springer.com/article/10.1007/BF02478259">propositional logic</a>.</p>
<p>We constructed a simple artificial neural network with just five neurons to perform a parity test. We gave the network signals between 0 and 40 pulses, which it classified as either odd or even. Despite its simplicity, the neural network correctly categorised the pulse numbers as odd or even with 100% accuracy.</p>
<p>This showed us that <em>in principle</em> parity categorisation does not require a large and complex brain such as a human’s. However, this doesn’t necessarily mean the bees and the simple neural network used the same mechanism to solve the task.</p>
<h2>Simple or complex?</h2>
<p>We don’t yet know how the bees were able to perform the parity task. Explanations may include simple or complex processes. For example, the bees may have:</p>
<ol>
<li><p>paired elements to find an unpaired element</p></li>
<li><p>performed division calculations – although division has not been previously demonstrated by bees</p></li>
<li><p>counted each element and then applied the odd/even categorisation rule to the total quantity.</p></li>
</ol>
<p>By teaching other animal species to discriminate between odd and even numbers, and perform other abstract mathematics, we can learn more about how maths and abstract thought emerged in humans.</p>
<p>Is discovering maths an inevitable consequence of intelligence? Or is maths somehow linked to the human brain? Are the differences between humans and other animals less than we previously thought? Perhaps we can glean these intellectual insights, if only we listen properly. </p>
<hr>
<p>
<em>
<strong>
Read more:
<a href="https://theconversation.com/how-a-bee-sees-tiny-bumps-on-flower-petals-give-them-their-intense-colour-and-help-them-survive-164782">How a bee sees: tiny bumps on flower petals give them their intense colour — and help them survive</a>
</strong>
</em>
</p>
<hr>
<img src="https://counter.theconversation.com/content/181040/count.gif" alt="The Conversation" width="1" height="1" />
<p class="fine-print"><em><span>Scarlett Howard received funding from Australian Government Research Training Program (RTP) Scholarship, RMIT University, Fyssen Foundation, L’Oreal-UNESCO for Women in Science Young Talents French Award, and Deakin University. She is affiliated with Pint of Science Australia as the Media Manager volunteer.</span></em></p><p class="fine-print"><em><span>Adrian Dyer receives funding from Australian Research Council</span></em></p><p class="fine-print"><em><span>Andrew Greentree receives funding from The Australian Research Council, Defence Science and Technology Group, SmartSat CRC, The US Air Force Office of Scientific Research, The Asian Office of Aerospace Research and Development, The US Office of Naval Research, and the Foundation for Australia-Japan Studies. </span></em></p><p class="fine-print"><em><span>Jair Garcia does not work for, consult, own shares in or receive funding from any company or organisation that would benefit from this article, and has disclosed no relevant affiliations beyond their academic appointment.</span></em></p>The miniature brains of honeybees were able to understand the concepts of odd and even, despite only having 960,000 neurons (compared to 86 billion in humans).Scarlett Howard, Lecturer, Monash UniversityAdrian Dyer, Associate Professor, RMIT UniversityAndrew Greentree, Professor of Quantum Physics and Australian Research Council Future Fellow, RMIT UniversityJair Garcia, Research fellow, RMIT UniversityLicensed as Creative Commons – attribution, no derivatives.tag:theconversation.com,2011:article/1291132020-02-13T17:41:46Z2020-02-13T17:41:46ZDesigning artificial brains can help us learn more about real ones<figure><img src="https://images.theconversation.com/files/311914/original/file-20200126-81346-ymrrb1.jpg?ixlib=rb-1.1.0&rect=18%2C0%2C6026%2C3471&q=45&auto=format&w=496&fit=clip" /><figcaption><span class="caption">Understanding how the computations in the brain go wrong could help scientists develop treatments for neurological disorders.</span> <span class="attribution"><span class="source">(Shutterstock)</span></span></figcaption></figure><p>Despite billions of dollars spent and decades of research, computation in the human brain remains largely a mystery. Meanwhile, we have made great strides in the development of artificial neural networks, which are designed to loosely mimic how brains compute. We have learned a lot about the nature of neural computation from these artificial brains and it’s time to take what we’ve learned and apply it back to the biological ones. </p>
<p><a href="https://doi.org/10.1016/S1474-4422(18)30499-X">Neurological diseases are on the rise worldwide</a>, making a better understanding of computation in the brain a pressing problem. Given the ability of modern artificial neural networks to solve complex problems, a framework for neuroscience guided by machine learning insights may unlock valuable secrets about our own brains and how they can malfunction.</p>
<p>Our thoughts and behaviours are generated by computations that take place in our brains. To effectively treat neurological disorders that alter our thoughts and behaviours, like schizophrenia or depression, we likely have to understand how the computations in the brain go wrong. </p>
<p>However, understanding neural computation has proven to be an immensely difficult challenge. When neuroscientists record activity in the brain, it is <a href="https://www.researchgate.net/publication/2889645_What_is_the_other_85_of_V1_doing">often indecipherable</a>. </p>
<p>In a paper published in <em>Nature Neuroscience</em>, my co-authors and I argue that the lessons we have learned from artificial neural networks can <a href="https://doi.org/10.1038/s41593-019-0520-2">guide us down the right path of understanding the brain as a computational system rather than as a collection of indecipherable cells</a>.</p>
<h2>Brain network models</h2>
<p>Artificial neural networks are computational models that loosely mimic the integration and activation properties of real neurons. They have become ubiquitous in the field of artificial intelligence.</p>
<p>To construct artificial neural networks, you start by first designing the network architecture: how the different components of the network are connected to one another. Then, you define the learning goal for the architecture, such as “learn to predict what you’re going to see next.” Then, you define a rule that tells the network how to change in order to achieve that goal using the data it receives.</p>
<p>What you do not do is specify how each neuron in the network is going to function. You leave it up to the network to determine how each neuron should function to best accomplish the task. I believe the development of the brain is probably the product of a similar process, <a href="https://doi.org/10.1145/3205651.3208249">both on an evolutionary timescale and at the timescale of an individual learning within their lifetime</a>.</p>
<figure class="align-center ">
<img alt="" src="https://images.theconversation.com/files/311915/original/file-20200126-81346-l1md44.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=754&fit=clip" srcset="https://images.theconversation.com/files/311915/original/file-20200126-81346-l1md44.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=600&h=337&fit=crop&dpr=1 600w, https://images.theconversation.com/files/311915/original/file-20200126-81346-l1md44.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=600&h=337&fit=crop&dpr=2 1200w, https://images.theconversation.com/files/311915/original/file-20200126-81346-l1md44.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=600&h=337&fit=crop&dpr=3 1800w, https://images.theconversation.com/files/311915/original/file-20200126-81346-l1md44.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=754&h=424&fit=crop&dpr=1 754w, https://images.theconversation.com/files/311915/original/file-20200126-81346-l1md44.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=754&h=424&fit=crop&dpr=2 1508w, https://images.theconversation.com/files/311915/original/file-20200126-81346-l1md44.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=754&h=424&fit=crop&dpr=3 2262w" sizes="(min-width: 1466px) 754px, (max-width: 599px) 100vw, (min-width: 600px) 600px, 237px">
<figcaption>
<span class="caption">Neuroscientists have mapped out the various regions of the brain, but how it computes remains a mystery.</span>
<span class="attribution"><span class="source">(Shutterstock)</span></span>
</figcaption>
</figure>
<h2>Assigning neuron roles</h2>
<p>This calls into question the usefulness of trying to determine the functions of individual neurons in the brain, when it is possible that these neurons are the result of an optimization process much like what we see with artificial neural networks.</p>
<p>The different components of artificial neural networks are often very hard to understand. There’s no simple verbal or simple mathematical description that explains exactly what they do.</p>
<p>In our paper, we propose that the same holds true for the brain, and so we have to move away from trying to understand the role of each neuron in the brain and instead look at the brain’s architecture, that is its network structure; the optimization goals, either at the evolutionary timescale or within the person’s lifetime; and the rules by which the brain updates itself — either over generations or within a lifetime — to meet those goals. By defining these three components, we may get a much better understanding of how the brain works than by trying to state what each neuron does. </p>
<h2>Optimizing frameworks</h2>
<p>One successful application of this approach has shown that the dopamine releasing neurons in the brain appear to encode information about unexpected rewards, e.g. unexpected delivery of some food. This sort of signal, called a reward prediction error, is often used to train artificial neural networks to maximize the rewards they get. </p>
<p>For example, by programming an artificial neural network to interpret points received in a video game as a reward, you can use reward prediction errors to train the network how to play the video game. In the real brain, as in the artificial neural networks, even if we don’t understand what each individual signal means, we can understand the role of these neurons and the neurons that receive their signals in relation to the learning goal of maximizing rewards.</p>
<figure class="align-center zoomable">
<a href="https://images.theconversation.com/files/311916/original/file-20200126-81352-hnx87k.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=1000&fit=clip"><img alt="" src="https://images.theconversation.com/files/311916/original/file-20200126-81352-hnx87k.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=754&fit=clip" srcset="https://images.theconversation.com/files/311916/original/file-20200126-81352-hnx87k.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=600&h=400&fit=crop&dpr=1 600w, https://images.theconversation.com/files/311916/original/file-20200126-81352-hnx87k.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=600&h=400&fit=crop&dpr=2 1200w, https://images.theconversation.com/files/311916/original/file-20200126-81352-hnx87k.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=600&h=400&fit=crop&dpr=3 1800w, https://images.theconversation.com/files/311916/original/file-20200126-81352-hnx87k.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=754&h=503&fit=crop&dpr=1 754w, https://images.theconversation.com/files/311916/original/file-20200126-81352-hnx87k.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=754&h=503&fit=crop&dpr=2 1508w, https://images.theconversation.com/files/311916/original/file-20200126-81352-hnx87k.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=754&h=503&fit=crop&dpr=3 2262w" sizes="(min-width: 1466px) 754px, (max-width: 599px) 100vw, (min-width: 600px) 600px, 237px"></a>
<figcaption>
<span class="caption">Neurological disorders are the second leading group cause of deaths in the world; artificial neural networks may help to understand their causes.</span>
<span class="attribution"><span class="source">(Shutterstock)</span></span>
</figcaption>
</figure>
<p>While current theories in systems neuroscience are beautiful and insightful, I believe a cohesive framework founded in the way in which evolution and learning shape our brain could fill in a lot the blanks we have been struggling with.</p>
<p>To make progress in systems neuroscience, it will take both bottom-up descriptive work, such as tracing out the connections and gene expression patterns of cells in the brain, and top-down theoretical work, using artificial neural networks to understand learning goals and learning rules. </p>
<p>Given the ability of modern artificial neural networks to solve complex problems, a framework for systems neuroscience guided by machine learning insights may unlock valuable secrets about the human brain.</p><img src="https://counter.theconversation.com/content/129113/count.gif" alt="The Conversation" width="1" height="1" />
<p class="fine-print"><em><span>Blake Richards receives funding from the Natural Sciences and Engineering Research Council of Canada, the Canadian Institute for Advanced Research, and Healthy Brains, Healthy Lives. </span></em></p>The knowledge produced in designing and developing artificial neural networks may provide new insights into how our brains work.Blake Richards, Assistant professor, Montreal Neurological Institute and the School of Computer Science, McGill UniversityLicensed as Creative Commons – attribution, no derivatives.tag:theconversation.com,2011:article/1108692019-07-09T11:23:44Z2019-07-09T11:23:44ZNeuroscience and artificial intelligence can help improve each other<figure><img src="https://images.theconversation.com/files/281257/original/file-20190625-81762-12028i3.jpg?ixlib=rb-1.1.0&rect=112%2C89%2C1181%2C938&q=45&auto=format&w=496&fit=clip" /><figcaption><span class="caption">Neurons treated with a fluorescent dye show their interconnections.</span> <span class="attribution"><span class="source">Silva Lab</span>, <a class="license" href="http://creativecommons.org/licenses/by-nd/4.0/">CC BY-ND</a></span></figcaption></figure><p>Despite their names, artificial intelligence technologies and their component systems, such as <a href="https://news.mit.edu/2017/explained-neural-networks-deep-learning-0414">artificial neural networks</a>, don’t have much to do with real brain science. I’m a <a href="https://www.silva.ucsd.edu">professor of bioengineering and neurosciences</a> interested in understanding how the brain works as a system – and how we can use that knowledge to design and engineer new machine learning models.</p>
<p>In recent decades, brain researchers have learned a huge amount about the physical connections in the brain and about how the nervous system routes information and processes it. But there is still a vast amount yet to be discovered. </p>
<p>At the same time, computer algorithms, software and hardware advances have brought machine learning to previously unimagined levels of achievement. <a href="https://scholar.google.com/citations?user=YkJ6RmUAAAAJ&hl=en">I</a> and other researchers in the field, including a number of <a href="https://www.axios.com/artificial-intelligence-pioneer-says-we-need-to-start-over-1513305524-f619efbd-9db0-4947-a9b2-7a4c310a28fe.html">its leaders</a>, have a growing sense that finding out more about <a href="https://doi.org/10.1016/j.neuron.2017.06.011">how the brain processes information</a> could <a href="https://medicalxpress.com/news/2017-08-neuroscience-advance-machine.html">help programmers translate</a> the concepts of thinking from the wet and squishy world of biology into all-new forms of machine learning in the digital world.</p>
<h2>The brain is not a machine</h2>
<p>“Machine learning” is one part of technologies that are often labeled “artificial intelligence.” Machine learning systems are better than humans at <a href="https://www.scientificamerican.com/article/ai-want-to-know-about-deep-learning-and-ai-check-this-out/">finding complex and subtle patterns</a> in very large data sets.</p>
<p>These systems seem to be everywhere – in self-driving cars, facial recognition software, financial fraud detection, robotics, helping with medical diagnoses and elsewhere. But under the hood, they’re all really just <a href="http://neuralnetworksanddeeplearning.com/chap2.html">variations on a single statistical-based algorithm</a>.</p>
<figure class="align-center zoomable">
<a href="https://images.theconversation.com/files/280294/original/file-20190619-171281-1chdj8g.png?ixlib=rb-1.1.0&q=45&auto=format&w=1000&fit=clip"><img alt="" src="https://images.theconversation.com/files/280294/original/file-20190619-171281-1chdj8g.png?ixlib=rb-1.1.0&q=45&auto=format&w=754&fit=clip" srcset="https://images.theconversation.com/files/280294/original/file-20190619-171281-1chdj8g.png?ixlib=rb-1.1.0&q=45&auto=format&w=600&h=536&fit=crop&dpr=1 600w, https://images.theconversation.com/files/280294/original/file-20190619-171281-1chdj8g.png?ixlib=rb-1.1.0&q=30&auto=format&w=600&h=536&fit=crop&dpr=2 1200w, https://images.theconversation.com/files/280294/original/file-20190619-171281-1chdj8g.png?ixlib=rb-1.1.0&q=15&auto=format&w=600&h=536&fit=crop&dpr=3 1800w, https://images.theconversation.com/files/280294/original/file-20190619-171281-1chdj8g.png?ixlib=rb-1.1.0&q=45&auto=format&w=754&h=673&fit=crop&dpr=1 754w, https://images.theconversation.com/files/280294/original/file-20190619-171281-1chdj8g.png?ixlib=rb-1.1.0&q=30&auto=format&w=754&h=673&fit=crop&dpr=2 1508w, https://images.theconversation.com/files/280294/original/file-20190619-171281-1chdj8g.png?ixlib=rb-1.1.0&q=15&auto=format&w=754&h=673&fit=crop&dpr=3 2262w" sizes="(min-width: 1466px) 754px, (max-width: 599px) 100vw, (min-width: 600px) 600px, 237px"></a>
<figcaption>
<span class="caption">A diagram of a simple artificial neural network.</span>
<span class="attribution"><a class="source" href="https://commons.wikimedia.org/wiki/File:Artificial_neural_network.svg">Cburnett/Wikimedia Commons</a>, <a class="license" href="http://creativecommons.org/licenses/by-sa/4.0/">CC BY-SA</a></span>
</figcaption>
</figure>
<p>Artificial neural networks, the most common mainstream approach to machine learning, are highly interconnected networks of digital processors that accept inputs, process measurements about those inputs and generate outputs. They need to learn what outputs should result from various inputs, until they develop the ability to respond to similar patterns in similar ways. </p>
<p>If you want a machine learning system to display the text “This is a cow” when it is shown a photo of a cow, you’ll first have to give it an enormous number of different photos of various types of cows from all different angles so it can adjust its internal connections in order to respond “This is a cow” to each one. If you show this system a photo of a cat, it will know only that it’s not a cow – and won’t be able to say what it actually is.</p>
<p>But that’s not how the brain learns, nor how it handles information to make sense of the world. Rather, the brain takes in a very small amount of input data – like a photograph of a cow and a drawing of a cow. Very quickly, and after only a very small number of examples, <a href="https://www.scientificamerican.com/article/gopnik-artificial-intelligence-helps-in-learning-how-children-learn/">even a toddler will grasp the idea</a> of what a cow looks like and be able to identify one in new images, from different angles and in different colors.</p>
<h2>But a machine isn’t a brain, either</h2>
<p>Because the brain and machine learning systems use fundamentally different algorithms, each excels in ways the other fails miserably. For instance, the brain can process information efficiently even when there is noise and uncertainty in the input – or under unpredictably changing conditions. </p>
<p>You could look at a grainy photo on ripped and crumpled paper, depicting a type of cow you had never seen before, and still think “that’s a cow.” Similarly, you routinely look at partial information about a situation and make predictions and decisions based on what you know, despite all that you don’t.</p>
<figure class="align-center zoomable">
<a href="https://images.theconversation.com/files/281438/original/file-20190626-76738-1pwnr27.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=1000&fit=clip"><img alt="" src="https://images.theconversation.com/files/281438/original/file-20190626-76738-1pwnr27.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=754&fit=clip" srcset="https://images.theconversation.com/files/281438/original/file-20190626-76738-1pwnr27.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=600&h=369&fit=crop&dpr=1 600w, https://images.theconversation.com/files/281438/original/file-20190626-76738-1pwnr27.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=600&h=369&fit=crop&dpr=2 1200w, https://images.theconversation.com/files/281438/original/file-20190626-76738-1pwnr27.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=600&h=369&fit=crop&dpr=3 1800w, https://images.theconversation.com/files/281438/original/file-20190626-76738-1pwnr27.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=754&h=464&fit=crop&dpr=1 754w, https://images.theconversation.com/files/281438/original/file-20190626-76738-1pwnr27.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=754&h=464&fit=crop&dpr=2 1508w, https://images.theconversation.com/files/281438/original/file-20190626-76738-1pwnr27.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=754&h=464&fit=crop&dpr=3 2262w" sizes="(min-width: 1466px) 754px, (max-width: 599px) 100vw, (min-width: 600px) 600px, 237px"></a>
<figcaption>
<span class="caption">Neuroscientists are still learning how things work inside even this small ‘mini-brain’ cluster of neurons and related cells.</span>
<span class="attribution"><a class="source" href="https://www.nsf.gov/news/mmg/mmg_disp.jsp?med_id=133641">Hoffman-Kim lab, Brown University/National Science Foundation</a></span>
</figcaption>
</figure>
<p>Equally important is the brain’s ability to recover from physical problems, reconfiguring its connections to adapt after an injury or a stroke. The brain is so impressive that patients with severe medical conditions can have <a href="https://www.encephalitis.info/rasmussens-encephalitis">as much as half of their brain removed</a> and recover normal cognitive and physical function. Now imagine how well a computer would work with half its circuits removed.</p>
<p>Equally impressive is the brain’s capability to make inferences and extrapolations, the keys to creativity and imagination. Consider the idea of a cow flipping burgers on Jupiter who at the same time is solving quantum gravity problems in its head. Neither of us has any experience of anything like that, but I can come up with it and efficiently communicate it to you, thanks to our brains.</p>
<p>Perhaps most astonishingly, though, the brain does all this with roughly the same <a href="https://www.scientificamerican.com/article/computers-vs-brains/">amount of power it takes to run a dim lightbulb</a>.</p>
<h2>Combining neuroscience and machine learning</h2>
<p>In addition to discovering how the brain works, it’s not at all clear which brain processes might work well as machine learning algorithms, or how to make that translation. One way to sort through all the possibilities is to focus on ideas that advance two research efforts at once, both improving machine learning and identifying new areas of neuroscience. Lessons can go both ways, from brain science to artificial intelligence – and back, with AI research highlighting new questions for biological neuroscientists.</p>
<figure class="align-center zoomable">
<a href="https://images.theconversation.com/files/281435/original/file-20190626-76726-1kyvbz0.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=1000&fit=clip"><img alt="" src="https://images.theconversation.com/files/281435/original/file-20190626-76726-1kyvbz0.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=754&fit=clip" srcset="https://images.theconversation.com/files/281435/original/file-20190626-76726-1kyvbz0.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=600&h=450&fit=crop&dpr=1 600w, https://images.theconversation.com/files/281435/original/file-20190626-76726-1kyvbz0.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=600&h=450&fit=crop&dpr=2 1200w, https://images.theconversation.com/files/281435/original/file-20190626-76726-1kyvbz0.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=600&h=450&fit=crop&dpr=3 1800w, https://images.theconversation.com/files/281435/original/file-20190626-76726-1kyvbz0.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=754&h=566&fit=crop&dpr=1 754w, https://images.theconversation.com/files/281435/original/file-20190626-76726-1kyvbz0.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=754&h=566&fit=crop&dpr=2 1508w, https://images.theconversation.com/files/281435/original/file-20190626-76726-1kyvbz0.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=754&h=566&fit=crop&dpr=3 2262w" sizes="(min-width: 1466px) 754px, (max-width: 599px) 100vw, (min-width: 600px) 600px, 237px"></a>
<figcaption>
<span class="caption">Neurons can grow in very complex shapes.</span>
<span class="attribution"><a class="source" href="https://www.shutterstock.com/image-illustration/pyramidal-neurons-found-cerebral-cortex-hippocampus-272003543">Juan Gaertner/Shutterstock.com</a></span>
</figcaption>
</figure>
<p>For example, in <a href="https://www.silva.ucsd.edu">my own lab</a>, we have developed a way to think about how individual neurons contribute to their overall network. Each neuron exchanges information only with the other specific neurons it is connected to. It has no overall concept of what the rest of the neurons are up to, or what signals they are sending or receiving. This is true for every neuron, no matter how broad the network, so local interactions collectively influence the activity of the whole.</p>
<p>It turns out that the mathematics that describe these layers of interaction are equally applicable to artificial neural networks and biological neural networks in real brains. As a result, we are developing a <a href="https://arxiv.org/abs/1804.07609">fundamentally new form of machine learning</a> that can learn on the fly without advance training that seems to be highly adaptable and efficient at learning.</p>
<p>In addition, we have used those ideas and mathematics to explore <a href="http://www.silva.ucsd.edu/news/2018/7/11/khgd49e0x6czgxlbf6c2jujnitt2kw">why the shapes of biological neurons are so twisted and convoluted</a>. We’ve found that they may develop those shapes to maximize their efficiency at passing messages, following the same computational rules we are using to build our artificial learning system. This was not a chance discovery we made about the neurobiology: We went looking for this relationship because the math told us to.</p>
<p>Taking a similar approach may also inform research into what happens when the brain falls prey to neurological and neurodevelopment disorders. Focusing on the principles and mathematics that AI and neuroscience share can help advance research into both fields, achieving new levels of ability for computers and understanding of natural brains.</p>
<p>[ <em><a href="https://theconversation.com/us/newsletters?utm_source=TCUS&utm_medium=inline-link&utm_campaign=newsletter-text&utm_content=expertise">Expertise in your inbox. Sign up for The Conversation’s newsletter and get a digest of academic takes on today’s news, every day.</a></em> ]</p><img src="https://counter.theconversation.com/content/110869/count.gif" alt="The Conversation" width="1" height="1" />
<p class="fine-print"><em><span>Gabriel A. Silva currently receives funding from Microsoft, Lawrence Livermore National Laboratory, and The Wholistic Research and Education Foundation. Past funding to his lab has come from the National Institutes of Health, the Department of Defense, and others. </span></em></p>Finding out more about how the brain works could help programmers translate thinking from the wet and squishy world of biology into all-new forms of machine learning in the digital world.Gabriel A. Silva, Professor of Bioengineering and Neurosciences; Founding Director, Center for Engineered Natural Intelligence, University of California, San DiegoLicensed as Creative Commons – attribution, no derivatives.tag:theconversation.com,2011:article/1191522019-07-01T12:57:32Z2019-07-01T12:57:32ZThe answer to forecasting Bitcoin may lie in artificial intelligence<figure><img src="https://images.theconversation.com/files/281497/original/file-20190627-76705-1m6c98s.jpg?ixlib=rb-1.1.0&rect=8%2C35%2C5982%2C2955&q=45&auto=format&w=496&fit=clip" /><figcaption><span class="caption">With no physical form, the cryptocurrency Bitcoin is difficult to analyse and its trading patterns challenging to discern.</span> <span class="attribution"><span class="source">Shutterstock</span></span></figcaption></figure><p>As <a href="https://www.bloomberg.com/news/articles/2019-06-26/bitcoin-tops-12-000-level-for-first-time-since-january-2018">Bitcoin tries to regain some of the lustre</a> it held in late 2017 when it nearly reached US$20,000 in value, investors are still questioning how to predict such a volatile currency.</p>
<p>As a cryptocurrency, there is no physical form that gives Bitcoin value, so it is impossible to perform traditional fundamental analysis of the currency. Consequently, many investors track the so-called <a href="https://files.stlouisfed.org/files/htdocs/publications/review/97/09/9709cn.pdf">technical trading indicators</a> (geometric patterns constructed from historical prices and trading volumes) in order to understand and predict Bitcoin’s future movement.</p>
<p>Some researchers have found success with <a href="https://ieeexplore.ieee.org/abstract/document/8125674">large complicated models</a>. But these sometimes have hundreds of variables (or predictors) and it is difficult to determine key factors or test the replicability of such approaches. It’s also hard to understand what factors really drive Bitcoin fluctuations on the market.</p>
<p>For over 20 years, I have been researching <a href="https://www.uoguelph.ca/lang/people/nikola-gradojevic">the applications of AI in finance</a>. At the Lang School of Business and Economics at the University of Guelph, my co-author and former graduate student Robert Adcock and I <a href="https://doi.org/10.1016/j.physa.2019.121727">created an artificial neural network (ANN) model</a> to test the predictability of Bitcoin prices. </p>
<h2>Predicting fluctuations</h2>
<p>We used technical indicators called moving averages as predictors. Moving averages are constructed by averaging prices over a period of time (e.g. 50 or 200 days) and plotting them as a line along with the prices. The rationale for using moving averages is that if the price of Bitcoin today becomes greater or lower than the average price over the past 50 or 200 days, traders could expect the emergence of an upward or downward trend. </p>
<p>If Bitcoin is unpredictable, then our model is not expected to beat the <a href="https://people.duke.edu/%7Ernau/411rand.htm">random walk</a> model — essentially, it is no better than guessing.</p>
<p>However, our <a href="https://doi.org/10.1016/j.physa.2019.121727">model provided some very interesting results</a> regarding Bitcoin’s predictability over time and during bouts of unusual volatility.</p>
<figure class="align-center zoomable">
<a href="https://images.theconversation.com/files/281859/original/file-20190628-94700-19beuxn.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=1000&fit=clip"><img alt="" src="https://images.theconversation.com/files/281859/original/file-20190628-94700-19beuxn.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=754&fit=clip" srcset="https://images.theconversation.com/files/281859/original/file-20190628-94700-19beuxn.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=600&h=401&fit=crop&dpr=1 600w, https://images.theconversation.com/files/281859/original/file-20190628-94700-19beuxn.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=600&h=401&fit=crop&dpr=2 1200w, https://images.theconversation.com/files/281859/original/file-20190628-94700-19beuxn.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=600&h=401&fit=crop&dpr=3 1800w, https://images.theconversation.com/files/281859/original/file-20190628-94700-19beuxn.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=754&h=503&fit=crop&dpr=1 754w, https://images.theconversation.com/files/281859/original/file-20190628-94700-19beuxn.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=754&h=503&fit=crop&dpr=2 1508w, https://images.theconversation.com/files/281859/original/file-20190628-94700-19beuxn.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=754&h=503&fit=crop&dpr=3 2262w" sizes="(min-width: 1466px) 754px, (max-width: 599px) 100vw, (min-width: 600px) 600px, 237px"></a>
<figcaption>
<span class="caption">Trading in bitcoin futures began Sunday, Dec. 10, 2017 on the Chicago Board Options Exchange. Here, traders work in a trading pit at the CBOE the Monday after.</span>
<span class="attribution"><span class="source">Kiichiro Sato/AP Photo</span></span>
</figcaption>
</figure>
<h2>Artificial intelligence forecasts</h2>
<p>Using daily observations from 2011-2018, we created an ANN with three predictors: returns, 50-day buy-sell signal and 200-day buy-sell signal.</p>
<p>We also tested an ANN model that included the <a href="https://www.investopedia.com/terms/v/vix.asp">Chicago Board Options Exchange Volatility Index (VIX)</a> to see if stock market volatility had any noticeable effect on Bitcoin movements. The <a href="https://www.forbes.com/sites/briankelly/2018/10/23/the-dollar-is-the-new-vix-and-the-bitcoin-connection/">VIX is an index providing theoretical 30-day market expectations</a> as based on the <a href="https://ca.spindices.com/indices/equity/sp-500">S&P 500 Index</a>. Higher values of VIX indicate that the market will make a large swing.</p>
<p>Artificial neural networks operate in a similar way to the basic functioning of the human brain. Our model takes predictors, or inputs, and outputs (the daily price change of Bitcoin) and attempts to learn a pattern from all the data. It continues to test its patterns until it reaches an optimal point where further testing is redundant. These advanced models form the backbone of many <a href="https://www.theglobeandmail.com/partners/advreinventingthefuture0516/why-you-can-bank-on-a-smarter-wall-street/article29822473/">AI learning programs that are used in business and engineering</a>.</p>
<p>By combining Bitcoin technical analysis and neural networks, we hoped that the ANN would find a pattern among the data that allowed us to more accurately predict future returns.</p>
<h2>Non-traditional investors</h2>
<p>Our ANN model did indeed succeed in reducing the prediction error of the random walk by about five to 10 per cent over the full observation period. These forecast improvements are statistically significant, indicating that predicting Bitcoin prices on a daily basis is no longer guesswork. Our results show that Bitcoin is unaffected by how the stock market changes, which suggests that traditional market investors and investors in Bitcoin are two distinct groups.</p>
<p>We also separated the data into four subsamples of similar time frames to further zoom in on market inefficiencies. Our ANN’s predictive performance improved further within these subsamples.</p>
<p>One subsample, running from October 2014 to June 2016, provided the best results of the study. The isolated 200-day signal model outperformed the random walk by 43.55 per cent. We noted that this subsample had low volatility compared to the other three subsamples and was the steadiest period of data we observed. In essence, greater market volatility makes learning data patterns and training of the ANN model more difficult.</p>
<p>Along with price accuracy, we also observed how often our ANN models correctly predicted whether prices would increase or decrease. Our main comprehensive model over the entire 2011-2018 period had nearly 63 per cent prediction accuracy. Put differently, Bitcoin trading with our model would be on average more profitable than placing random buy and sell orders that have a 50 per cent chance of making a profit.</p>
<h2>Speculation and predictive bubbles</h2>
<p>Compared to other predictive models, our ANN provided the most accurate and reliable predictive method for Bitcoin. We concluded that the historical evolution of daily Bitcoin prices followed predictive trends (or bubbles) that potentially arise from the speculative nature of cryptocurrency trading.</p>
<hr>
<p>
<em>
<strong>
Read more:
<a href="https://theconversation.com/how-low-will-bitcoin-now-go-the-history-of-price-bubbles-provides-some-clues-107596">How low will Bitcoin now go? The history of price bubbles provides some clues</a>
</strong>
</em>
</p>
<hr>
<p>We believe that the future of forecasting Bitcoin — and perhaps investing in general — lies in the abilities of artificial intelligence and artificial neural networks. While people may argue over the merits of Bitcoin as a currency, we can at least appreciate it as a fascinating — and now easier-to-predict — commodity.</p><img src="https://counter.theconversation.com/content/119152/count.gif" alt="The Conversation" width="1" height="1" />
<p class="fine-print"><em><span>Nikola Gradojevic does not work for, consult, own shares in or receive funding from any company or organisation that would benefit from this article, and has disclosed no relevant affiliations beyond their academic appointment.</span></em></p>Bitcoin trading is difficult to predict, but artificial neural networks may be able to discern patterns.Nikola Gradojevic, Professor, Finance, University of GuelphLicensed as Creative Commons – attribution, no derivatives.