tag:theconversation.com,2011:/africa/topics/wisdom-of-crowds-24461/articlesWisdom of Crowds – The Conversation2023-11-20T18:59:52Ztag:theconversation.com,2011:article/2175392023-11-20T18:59:52Z2023-11-20T18:59:52ZDisinformation campaigns are undermining democracy. Here’s how we can fight back<p>Misinformation is debated everywhere and has justifiably sparked concerns. It can polarise the public, reduce health-protective behaviours such as mask wearing and vaccination, and erode trust in science. Much of misinformation is spread not by accident but as <a href="https://www.nature.com/articles/nclimate2875">part of organised political campaigns</a>, in which case we refer to it as disinformation. </p>
<p>But there is a more fundamental, subversive damage arising from misinformation and disinformation that is discussed less often.</p>
<p>It undermines democracy itself. In a recent <a href="https://www.sciencedirect.com/science/article/pii/S2352250X23001562">paper published in Current Opinion in Psychology</a>, we highlight two important aspects of democracy that disinformation works to erode.</p>
<h2>The integrity of elections</h2>
<p>The first of the two aspects is confidence in how power is distributed – the integrity of elections in particular.</p>
<p>In the United States, recent polls have shown nearly 70% of Republicans <a href="https://www.cnn.com/2023/08/03/politics/cnn-poll-republicans-think-2020-election-illegitimate/index.html">question the legitimacy of the 2020 presidential election</a>. This is a direct result of <a href="https://www.cambridge.org/core/journals/perspectives-on-politics/article/donald-trump-and-the-lie/A438DF5A45FE78CB2BC887859EFAB587">disinformation from Donald Trump</a>, the loser of that election. </p>
<p>Democracy depends on the people knowing that power will be transferred peacefully <a href="https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3273111">if an incumbent loses an election</a>. The “big lie” that the 2020 US election was stolen undermines that confidence.</p>
<h2>Depending on reliable information</h2>
<p>The second important aspect of democracy is this – it depends on reliable information about the evidence for various policy options.</p>
<p>One reason we trust democracy as a system of governance is the idea that it can deliver “better” decisions and outcomes than autocracy, because the “wisdom of crowds” <a href="https://onlinelibrary.wiley.com/doi/full/10.1111/tops.12610">outperforms any one individual</a>. But the <a href="https://www.tandfonline.com/doi/full/10.1080/08913811.2018.1575007">benefits of this wisdom vanish</a> if people are pervasively disinformed. </p>
<p>Disinformation about climate change is a well-documented example. The fossil fuel industry understood the environmental consequences of burning fossil fuels <a href="https://www.nature.com/articles/s41558-018-0349-9">at least as early as the 1960s</a>. Yet they spent decades funding organisations that <a href="https://www.nature.com/articles/nclimate2875">denied the reality of climate change</a>. This disinformation campaign has delayed climate mitigation by several decades – a case of public policy being thwarted by false information.</p>
<p>We’ve seen a similar misinformation trajectory in the COVID-19 pandemic, although it happened in just a few years rather than decades. Misinformation about COVID varied from claims that 5G towers rather than a virus <a href="https://journals.sagepub.com/doi/10.1177/1329878X20946113">caused the disease</a>, to casting doubt on the <a href="https://www.cmaj.ca/content/195/15/E552">effectiveness of lockdowns</a> or the <a href="https://www.nature.com/articles/s41562-023-01644-3">safety of vaccines</a>.</p>
<p>The viral surge of misinformation led to the World Health Organisation introducing a new term – <a href="https://www.sciencedirect.com/science/article/pii/S014067362030461X">infodemic</a> – to describe the abundance of low-quality information and conspiracy theories.</p>
<hr>
<p>
<em>
<strong>
Read more:
<a href="https://theconversation.com/its-almost-like-grooming-how-anti-vaxxers-conspiracy-theorists-and-the-far-right-came-together-over-covid-168383">'It's almost like grooming': how anti-vaxxers, conspiracy theorists, and the far-right came together over COVID</a>
</strong>
</em>
</p>
<hr>
<h2>A common denominator of misinformation</h2>
<p>Strikingly, some of the same political operatives involved in denying climate change have also used their rhetorical playbook <a href="https://www.lse.ac.uk/granthaminstitute/news/organisers-of-anti-lockdown-declaration-have-track-record-of-promoting-denial-of-health-and-environmental-risks/">to promote COVID disinformation</a>. What do these two issues have in common?</p>
<p>One common denominator is suspicion of government solutions to societal problems. Whether it’s setting a price on carbon to mitigate climate change, or social distancing to slow the spread of COVID, contrarians fear the policies they consider to be <a href="http://refhub.elsevier.com/S2352-250X(23)00156-2/sref46">an attack on personal liberties</a>.</p>
<p>An ecosystem of conservative and free-market think tanks exists to deny any science that, if acted on, has the potential to infringe on “liberty” through regulations.</p>
<p>There is another common attribute that ties together all organised disinformation campaigns – whether about elections, climate change or vaccines. It’s the use of personal attacks to compromise people’s integrity and credibility.</p>
<p>Election workers in the US <a href="https://edition.cnn.com/2023/08/30/politics/rudy-giuliani-georgia-election-workers/index.html">were falsely accused</a> of committing fraud by those who fraudulently claimed the election had been “stolen” from Trump.</p>
<p>Climate scientists have been subject to <a href="https://jspp.psychopen.eu/index.php/jspp/article/view/4965">harassment campaigns</a>, ranging from hate mail to vexatious complaints and freedom-of-information requests. Public health officials such as Anthony Fauci have been <a href="https://link.springer.com/article/10.1057/s42738-021-00073-2">prominent targets of far-right attacks</a>.</p>
<h2>The new frontier in attacks on scientists</h2>
<p>It is perhaps unsurprising there is now a new frontier in the attacks on scientists and others who seek to uphold the evidence-based integrity of democracy. It involves attacks and allegations of bias against misinformation researchers. </p>
<p>Such attacks are largely <a href="https://www.newyorker.com/magazine/2023/10/30/jim-jordans-conspiratorial-quest-for-power">driven by Republican politicians</a>, in particular those who have endorsed Trump’s baseless claims about the 2020 election.</p>
<p>The misinformers are seeking to neutralise research focused on their own conduct by borrowing from the climate denial and anti-vaccination playbook. Their campaign has had a chilling effect <a href="https://www.washingtonpost.com/politics/2023/09/25/gop-legal-attacks-create-chilling-effect-misinformation-research/">on research into misinformation</a>.</p>
<hr>
<p>
<em>
<strong>
Read more:
<a href="https://theconversation.com/inoculate-yourself-against-election-misinformation-campaigns-3-essential-reads-193582">Inoculate yourself against election misinformation campaigns – 3 essential reads</a>
</strong>
</em>
</p>
<hr>
<h2>How do we move on from here?</h2>
<p>Psychological research has contributed to <a href="https://econtent.hogrefe.com/doi/10.1027/1016-9040/a000493">legislative efforts by the European Union</a>, such as the Digital Services Act or Code of Practice, which seek to make democracies more resilient against misinformation and disinformation. </p>
<p>Research has also investigated how to boost the public’s resistance to misinformation. One such method is <a href="https://dx.doi.org/10.1126/sciadv.abo6254">inoculation</a>, which rests on the idea people can be protected against being misled if they learn about the rhetorical techniques used to mislead them. </p>
<p>In a recent inoculation campaign involving brief educational videos shown to 38 million citizens in Eastern Europe, <a href="https://safety.google/intl/en_uk/stories/defanging-disinformation-in-CEE/">people’s ability to recognise misleading rhetoric</a> about Ukrainian refugees was frequently improved.</p>
<p>It remains to be seen whether these initiatives and research findings will be put to use in places like the US, where one side of politics appears more threatened by research into misinformation than by the risks to democracy arising from misinformation itself.</p>
<hr>
<p><em>We’d like to acknowledge our colleagues Ullrich Ecker, Naomi Oreskes, Jon Roozenbeek and Sander van der Linden who coauthored the journal article on which this article is based.</em></p><img src="https://counter.theconversation.com/content/217539/count.gif" alt="The Conversation" width="1" height="1" />
<p class="fine-print"><em><span>Stephan Lewandowsky receives financial support from the European Research Council, the Humboldt Foundation, the Volkswagen Foundation and the European Commission. He also receives funding from Jigsaw (a technology incubator created by Google) and from UK Research and Innovation. He also interacts frequently with the European Commission's Joint Research Centre in an advisory capacity and through scientific collaborations.</span></em></p><p class="fine-print"><em><span>John Cook does not work for, consult, own shares in or receive funding from any company or organisation that would benefit from this article, and has disclosed no relevant affiliations beyond their academic appointment.</span></em></p>When people are pervasively disinformed, the very foundations of democracy can end up on shaky ground.Stephan Lewandowsky, Chair of Cognitive Psychology, University of BristolJohn Cook, Senior Research Fellow, Melbourne School of Psychological Sciences, The University of MelbourneLicensed as Creative Commons – attribution, no derivatives.tag:theconversation.com,2011:article/1913172022-09-28T12:32:14Z2022-09-28T12:32:14ZNobel Prizes, election outcomes and sports championships – prediction markets try to foresee the future<figure><img src="https://images.theconversation.com/files/486954/original/file-20220928-12-t78jok.jpg?ixlib=rb-1.1.0&rect=0%2C122%2C4184%2C3075&q=45&auto=format&w=496&fit=clip" /><figcaption><span class="caption">Who will be next to cross this stage and accept a Nobel Prize?</span> <span class="attribution"><a class="source" href="https://www.gettyimages.com/detail/news-photo/general-view-of-the-stage-during-the-nobel-prize-awards-news-photo/500796996">Pascal Le Segretain/Getty Images News via Getty Images</a></span></figcaption></figure><p>Who will win Nobel Prizes in 2022? Wikipedia posits a handful of contenders for <a href="https://en.wikipedia.org/wiki/2022_Nobel_Prize_in_Physiology_or_Medicine">Physiology or Medicine</a>, about 20 different possible winners for the <a href="https://en.wikipedia.org/wiki/2022_Nobel_Peace_Prize">Peace Prize</a> and several dozen potential winners of the <a href="https://en.wikipedia.org/wiki/2022_Nobel_Prize_in_Literature">Literature Prize</a>. But since the Swedish Academy never announces nominees in advance, there are few insights indicating who will win, or even if the eventual winner is on a given list.</p>
<p>Are there ways to predict the future winners?</p>
<p>The <a href="https://apps.dtic.mil/sti/citations/AD0690498">Delphi approach</a>, named after the oracle in ancient Greece, gathers multiple rounds of opinions from a group of experts <a href="https://doi.org/10.1016/j.dss.2013.07.001">to generate a prediction</a>. Gambling firms provide betting odds on the likelihood that specific competitors will win. <a href="https://doi.org/10.1016/j.ejor.2016.12.043">Crowdsourced competitions</a>, such as the Yahoo Soccer World Cup “Pick-Em,” have participants predict individual contest winners and then aggregate the results.</p>
<p>Another approach is a prediction market that provides insight into what people expect will happen in the future by creating a stock market-like environment to capture the “<a href="https://www.penguinrandomhouse.com/books/175380/the-wisdom-of-crowds-by-james-surowiecki/">wisdom of the crowd</a>.” <a href="https://doi.org/10.1287/mnsc.44.8.1049">Groups and crowds often are collectively smarter</a> than individuals when many independent opinions are combined. </p>
<p>As an accounting and information systems professor at the University of Southern California, <a href="https://scholar.google.com/citations?user=DVk7EKAAAAAJ&hl=en&oi=ao">I investigate issues related to the crowd</a> both in my research and in my teaching. Here’s how prediction markets harness what the crowd thinks to forecast the future.</p>
<h2>The wisdom of the market</h2>
<p>In prediction markets, participants buy and sell stocks. Each stock’s price is tied to a different event happening in the future. Information about the future is captured in the stock prices. </p>
<p>For instance, in a prediction market focused on the Nobel Peace Prize, maybe Greta Thunberg is trading at $0.10 while Pope Francis is trading at $0.15, and the stocks for the entire group of candidates add up to sum to $1. The prices reflect the traders’ aggregated beliefs about the probability of their winning – a higher price means a higher perceived likelihood of winning.</p>
<p><iframe id="S3IU3" class="tc-infographic-datawrapper" src="https://datawrapper.dwcdn.net/S3IU3/3/" height="400px" width="100%" style="border: none" frameborder="0"></iframe></p>
<p>Prediction markets have various ways of setting stock prices. The Iowa Electronic Markets took following approach during the 2020 U.S. presidential election:</p>
<ul>
<li>Stock DEM2020 pays off $1 if the Democratic candidate wins, and $0 otherwise,</li>
<li>Stock REP2020 pays off $1 if the Republican candidate wins, and $0 otherwise.</li>
</ul>
<p>The stock prices capture the probabilities of each candidate winning, in two mutually exclusive events. If the price of DEM2020 is $0.52, then that is treated as the probability of that event occurring – a 52% chance. If DEM2020 is $0.52, then REP2020 is $0.48.</p>
<p>Prediction markets may use real money, or they can use play money. Google’s market used what it called “Goobles,” while the Hollywood Stock Exchange uses Hollywood Dollars. The Iowa Electronic Markets and PredictIt, both sponsored by universities, use real money. Researchers have found that there are <a href="https://doi.org/10.1080/10196780500491303">no differences in the performance of markets</a> using real money versus those using play money.</p>
<p>Although using play money makes it possible for many people to participate, one potential challenge for prediction markets that don’t use real money is <a href="https://doi.org/10.1016/j.dss.2015.07.004">gaining and maintaining interested participants</a>. Despite using different devices to keep up engagement, such as leader boards indicating who has accumulated the biggest portfolio, there is literally no money on the table to keep participants interested in the market.</p>
<figure class="align-center zoomable">
<a href="https://images.theconversation.com/files/486959/original/file-20220928-12981-49d7y1.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=1000&fit=clip"><img alt="German players hold 2014 World Cup trophy aloft" src="https://images.theconversation.com/files/486959/original/file-20220928-12981-49d7y1.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=754&fit=clip" srcset="https://images.theconversation.com/files/486959/original/file-20220928-12981-49d7y1.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=600&h=409&fit=crop&dpr=1 600w, https://images.theconversation.com/files/486959/original/file-20220928-12981-49d7y1.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=600&h=409&fit=crop&dpr=2 1200w, https://images.theconversation.com/files/486959/original/file-20220928-12981-49d7y1.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=600&h=409&fit=crop&dpr=3 1800w, https://images.theconversation.com/files/486959/original/file-20220928-12981-49d7y1.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=754&h=514&fit=crop&dpr=1 754w, https://images.theconversation.com/files/486959/original/file-20220928-12981-49d7y1.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=754&h=514&fit=crop&dpr=2 1508w, https://images.theconversation.com/files/486959/original/file-20220928-12981-49d7y1.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=754&h=514&fit=crop&dpr=3 2262w" sizes="(min-width: 1466px) 754px, (max-width: 599px) 100vw, (min-width: 600px) 600px, 237px"></a>
<figcaption>
<span class="caption">Market participants who know more about the game might better predict winners.</span>
<span class="attribution"><a class="source" href="https://newsroom.ap.org/detail/SoccerFIFAWorldCupFormat/f538c07d27f542cfaacbada7906cdacf/photo?Query=Germany%20World%20Cup%202014&mediaType=photo&sortBy=arrivaldatetime:desc&dateRange=&totalCount=7375&currentItemNo=40">AP Photo/Natacha Pisarenko</a></span>
</figcaption>
</figure>
<h2>Participants bring their knowledge to the market</h2>
<p>Prediction markets and crowdsourcing do not function in a vacuum. </p>
<p>Researchers have found that information about events finds its way into the prediction processes from various sources. For example, when I <a href="https://doi.org/10.1016/j.ejor.2016.12.043">analyzed the relationship between the betting odds</a> and the Yahoo Pick-Em crowd’s guesses for the 2014 FIFA World Cup, I found that there was no statistical difference between the proportion of correct guesses in each. My conclusion is that either the crowd’s guesses incorporated the betting odds information or the crowd’s guesses added up to the same result by some other means.</p>
<p>Generally, prediction markets use play money or are run by non-profit universities to study markets, elections and human decision making. Although gambling houses can take bets for many activities, <a href="https://www.legalsportsreport.com/74880/end-of-predictit-election-betting-around-the-corner/">external prediction markets are more restricted</a> in the activities they can be used to investigate, and are typically limited to elections. However, internal prediction markets – run within a corporation, for instance – <a href="https://doi.org/10.1007/978-3-319-07179-4_26">can explore almost any topic of interest</a>.</p>
<p>Typically, <a href="https://doi.org/10.1108/S1477-4070(2011)0000008014">prediction markets function better with informed participants</a>. Although using so-called inside information is illegal in some markets, including the New York Stock Exchange, there generally are no such limitations in prediction markets, or other crowdsourcing approaches. If those with inside information were to participate in a prediction market, it would likely lead to more accurate stock prices, as insiders make trades informed by their knowledge. However, if others <a href="https://doi.org/10.1016/j.accinf.2012.02.003">find out that a participant has inside information</a>, then they may very well try to gain access to that info, follow the insider’s actions or even decide to leave the unfair market.</p>
<p>The <a href="https://doi.org/10.1257/0895330041371321">accuracy of prediction markets</a> depends on many factors, including who is in the market, what their biases are and how heterogeneous the participants are. Accuracy can also depend on how many people are in the market – more is generally better – and the extent to which they are informed about the events of interest.</p>
<p>Researchers have found that prediction markets have <a href="https://doi.org/10.1016/j.ijforecast.2008.03.007">outperformed polls in presidential elections</a> roughly 75% of the time. But accurate results are not guaranteed. For example, <a href="https://www.bloomberg.com/opinion/articles/2016-11-15/prediction-markets-didn-t-call-trump-s-win-either">prediction markets did not correctly predict</a> that Donald Trump would win the U.S. presidency in 2016.</p>
<h2>Who will be in Stockholm for the ceremony?</h2>
<p>In 2011, Harvard University economics faculty had a real-money prediction market site, referred to as “the world’s most accurate prediction market.” The site had been used for predicting the Nobel Prize in Economics, but <a href="https://freakonomics.com/2011/10/harvard-shuts-down-its-nobel-prize-pool/">Harvard advised the site to shut down</a>.</p>
<p>I couldn’t find any current public prediction markets active for the 2022 Nobel Prizes.</p>
<p>For the moment, perhaps the closest to participating in a Nobel prediction market would be to place a bet at one of the gambling houses that <a href="https://www.gamblingsites.com/entertainment-betting/nobel-prize/">takes bets on the Nobel Prizes</a>. Or find a Nobel Prize Pick-Em site, propose such an event to an existing prediction market or build your own prediction market using <a href="https://www.cipher-sys.com/market-research-services">some of the available software</a>.</p>
<p>If you know of one, let me know, I want to play.</p><img src="https://counter.theconversation.com/content/191317/count.gif" alt="The Conversation" width="1" height="1" />
<p class="fine-print"><em><span>Daniel O'Leary does not work for, consult, own shares in or receive funding from any company or organization that would benefit from this article, and has disclosed no relevant affiliations beyond their academic appointment.</span></em></p>Buying and selling stocks – with real or play money – is a way to harness the wisdom of the crowd about questions like who is going to win a competition.Daniel O'Leary, Professor of Accounting and Information Systems, University of Southern CaliforniaLicensed as Creative Commons – attribution, no derivatives.tag:theconversation.com,2011:article/1627212021-06-24T13:55:16Z2021-06-24T13:55:16ZPooling society’s collective intelligence helped fight COVID – it must help fight future crises too<figure><img src="https://images.theconversation.com/files/407979/original/file-20210623-23-14h4dsl.jpeg?ixlib=rb-1.1.0&rect=6%2C140%2C4243%2C2440&q=45&auto=format&w=496&fit=clip" /><figcaption><span class="caption">
</span> <span class="attribution"><a class="source" href="https://www.shutterstock.com/image-photo/close-miniature-people-social-network-diagram-509324473">Wisiel/Shutterstock</a></span></figcaption></figure><p>A <a href="https://www.gov.uk/government/news/pm-announces-plan-for-global-pandemic-radar">Global Pandemic Radar</a> is to be created to detect new COVID variants and other emerging diseases. Led by the WHO, the project aims to build an international network of surveillance hubs, set up to share data that’ll help us monitor vaccine resistance, track diseases and identify new ones as they emerge.</p>
<p>This is undeniably a good thing. Perhaps more than any event in recent memory, the COVID pandemic has brought home the importance of pooling society’s collective intelligence and finding new ways to share that combined knowledge as quickly as possible.</p>
<p>At its simplest, <a href="https://www.nesta.org.uk/report/future-minds-and-machines/3-what-collective-intelligence/#content">collective intelligence</a> is the enhanced capacity that’s created when diverse groups of people work together, often with the help of <a href="https://theconversation.com/big-data-brings-new-power-to-open-source-intelligence-26554">technology</a>, to mobilise more information, ideas and knowledge to solve a problem. Digital technologies have transformed what can be achieved through collective intelligence in recent years – connecting more of us, <a href="https://www.nesta.org.uk/report/future-minds-and-machines/">augmenting human intelligence</a> with machine intelligence, and helping us to generate new insights from novel sources of data.</p>
<figure>
<iframe src="https://player.vimeo.com/video/391211147" width="500" height="281" frameborder="0" webkitallowfullscreen="" mozallowfullscreen="" allowfullscreen=""></iframe>
</figure>
<p>So what have we learned over the last 18 months of collective intelligence pooling that can inform the Global Pandemic Radar? Building from the COVID crisis, what lessons will help us perfect disease surveillance and respond better to future crises?</p>
<h2>People want to help scientists</h2>
<p>Responding to new and emerging threats requires new methods for filling data and evidence gaps fast. Collective intelligence methods like citizen science have been widely used in the <a href="https://www.tcv.org.uk/wp-content/uploads/2014/11/citizen-science-and-the-environment.pdf">environmental sector</a> for years, but savvy scientists quickly saw the opportunity to deploy these and other approaches to tap into the public’s appetite to contribute to the COVID-19 response. </p>
<p>Before doctors had access to mass community testing or accurate forecasting, for instance, data provided by the public was a valuable early source of information. For example, researchers at King’s College London quickly developed the <a href="https://covid.joinzoe.com/">COVID Zoe symptom tracker app</a>, to which over 4.6 million people have contributed their symptoms since March 2020. This data played a critical role in helping us understand how the virus affects different groups of people, exposing the <a href="https://covid.joinzoe.com/post/the-20-symptoms-of-covid-19-to-watch-out-for">variety</a> of COVID-19 symptoms people have experienced.</p>
<p>Even gamers have played their part behind the scenes. <a href="https://www.eveonline.com/discovery">Project Discovery</a> is described as a citizen science “mini-game”, in which gamers explore outer space while drawing polygons around clusters of cells. The cell populations they trace around are from flow cytometry data that would ordinarily be painstakingly pored over by scientists to see how a COVID infection affects different types of cell. Over 327,000 gamers have taken part since June 2020, saving scientists an estimated <a href="https://www.ccpgames.com/news/2021/eve-onlines-project-discovery-wins-webby-peoples-voice-award-for-public">330 years of research</a>.</p>
<figure>
<iframe width="440" height="260" src="https://www.youtube.com/embed/QygAfOksOZk?wmode=transparent&start=0" frameborder="0" allowfullscreen=""></iframe>
</figure>
<p>Perhaps more visibly, vaccine development efforts have also been fuelled by volunteers. Over <a href="https://digital.nhs.uk/dashboards/coronavirus-covid-19-vaccine-studies-volunteers-dashboard-uk#dashboard">500,000</a> people signed up to the UK’s COVID vaccine studies volunteer service.</p>
<p>Scientific training and research funding is not usually geared towards public participation and collaboration. That means, despite the potential, the public is typically excluded from participation in scientific research. Changing this might help us shift the dial on preventing the next pandemic and tackling a whole host of our other <a href="https://www.theguardian.com/education/2020/nov/16/the-rise-of-citizen-science-can-the-public-help-solve-our-biggest-problems">complex challenges</a>, such as climate change. </p>
<h2>Making sense of too much data</h2>
<p>Alongside this rise in citizen science, 2020 was also a bumper year for scientific research, seeing a <a href="https://www.google.com/url?q=http://doi.org/10.2139/ssrn.3712813&sa=D&source=editors&ust=1624473166605000&usg=AOvVaw2IyJwAH6kyjG_ZzrzRUFoR">15% increase</a> in paper submissions. <a href="https://reports.dimensions.ai/covid-19/">Over 475,000</a> COVID-related papers and pre-prints have been shared online as of June 2021. </p>
<p>This feverish scientific reporting, especially intense in the field of health and medicine, has raised concerns about quality control. <a href="https://www.nature.com/articles/d41586-020-03564-y">Traditional processes</a> of peer review have come under strain, with papers increasingly released as <a href="https://theconversation.com/how-the-conversation-handles-pre-prints-155063">pre-prints</a>, before they’ve been peer reviewed. Meanwhile, decision-makers face the challenge of finding the most relevant resources in the face of information overload. </p>
<hr>
<p>
<em>
<strong>
Read more:
<a href="https://theconversation.com/how-the-conversation-handles-pre-prints-155063">How The Conversation handles pre-prints</a>
</strong>
</em>
</p>
<hr>
<p>The collaborative health evidence database, <a href="https://www.epistemonikos.org/en/about_us/updated_report">Epistemonikos</a>, offers some relief to these challenges. It uses a combination of machine learning algorithms and crowd validation to identify all of the clinical systematic reviews related to the search query entered by the user. </p>
<p>In the past, it was used by policymakers in Chile to accelerate the process of public health legislation. Since 2020, the team behind Epistemonikos has identified <a href="https://app.iloveevidence.com/loves/5e6fdb9669c00e4ac072701d?utm=epdb_en">more than 6,000</a> systematic reviews related to COVID-19 within their database. Highlighting these has helped health professionals and decision-makers find what they’re looking for amid the noise.</p>
<p><div data-react-class="Tweet" data-react-props="{"tweetId":"1347623661356937224"}"></div></p>
<p>It isn’t just scientific research that has proved difficult to make sense of. The flood of data about the pandemic has also required careful collation, seeing as it often comes from multiple sources and is scattered across different websites and open databases, many of which follow different standards and formats. Data about a crisis is only useful if it’s synthesised and presented in ways that decision makers can understand.</p>
<p>One retrospective study showed how <a href="https://link.springer.com/article/10.1007/s13278-021-00723-5">Google searches</a> involving pandemic-related keywords, like “pneumonia”, could have been used to spot the early warning signs of COVID-19 spreading in Europe. The same finding was reached using <a href="https://www.nature.com/articles/s41598-021-81333-1">Twitter</a> data, and could in the future be reached with data from <a href="https://www.thelancet.com/journals/landig/article/PIIS2589-7500(19)30222-5/fulltext">wearable technology</a>. For now, these novel sources of data aren’t integrated into wider surveillance efforts, but doing so could help governments get better at anticipating crises in the future. </p>
<hr>
<p>
<em>
<strong>
Read more:
<a href="https://theconversation.com/big-data-can-help-doctors-predict-which-covid-patients-will-become-seriously-ill-153168">Big data can help doctors predict which COVID patients will become seriously ill</a>
</strong>
</em>
</p>
<hr>
<p>In the US, the absence of a publicly available system for aggregating COVID-related data led to the creation of the <a href="https://covidtracking.com/">COVID Tracking Project</a>. A community of over 300 volunteers collected, curated and analysed data sources to produce the most comprehensive public source of information about COVID in the US. Their efforts helped process under-reported data on those in <a href="https://covidtracking.com/nursing-homes-long-term-care-facilities">long-term care</a> and the incidence of COVID organised by <a href="https://covidtracking.com/race">race and ethnicity</a>.</p>
<p>However, another <a href="https://www.wsj.com/articles/data-driven-website-aims-to-help-global-agencies-make-decisions-on-coronavirus-pandemic-11595343601">promising pandemic initiative</a>, the Collective and Augmented Intelligence Against COVID-19 (CAIAC) project, <a href="http://caiac19.org/">failed</a> to get off the ground, despite the support of UNESCO and the Stanford Institute for Human-Centered Artificial Intelligence. The lesson: productively combining human and machine intelligence could help us deal with overwhelming amounts of data, but it isn’t easy. Creating and maintaining new global data infrastructures takes time, effort and significant investment.</p>
<h2>Diversity enhances collective intelligence</h2>
<p>There’s more we can do to properly harness collective intelligence when facing future crises. More data certainly helps, and those who organise that data can help thrust it before key decision makers as quickly as possible. But who makes the decisions matters too.</p>
<p>With the world taken by surprise, it seems that COVID-19 decision making followed the usual modus operandi of excluding the voices of women and minorities. An <a href="https://gh.bmj.com/content/5/10/e003549">analysis</a> of 115 COVID-19 decision-making and expert task forces from 87 countries, including the UK and the US, found that just 3.5% had gender parity in their membership, while 85.2% were majority men. Would the disproportionate impact of COVID-19 on <a href="https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/892376/COVID_stakeholder_engagement_synthesis_beyond_the_data.pdf">black and ethnic minority</a> communities and <a href="https://www.weforum.org/agenda/2020/09/covid-19-gender-inequality-jobs-economy/">women</a> have been as severe if these expert groups had been more diverse? </p>
<hr>
<p>
<em>
<strong>
Read more:
<a href="https://theconversation.com/49-more-likely-to-die-racial-inequalities-of-covid-19-laid-bare-in-study-of-east-london-hospitals-153834">'49% more likely to die' – racial inequalities of COVID-19 laid bare in study of East London hospitals</a>
</strong>
</em>
</p>
<hr>
<p>The collective intelligence literature has long pointed to <a href="https://www.sciencedirect.com/science/article/abs/pii/S0749597816303193">the potential of diversity</a> in problem solving, but these positive effects can only be realised if institutions actively seek out a variety of voices. Without finding better ways to bring diverse perspectives into <a href="https://www.nesta.org.uk/report/how-make-good-group-decisions/">decision making</a>, we’re not going to get too excited about how equally the benefits of the Global Pandemic Radar, and other future efforts to pool data and intelligence, will be felt. </p>
<p>While COVID has elevated AI-enabled modelling to the heart of government decisions, there is still a long way to go before these models are accessible to ordinary people - something which could help diversify decision making. This is where more creative participatory methods, aimed at helping members of the public explore the consequences of policy decisions and collective behaviours, may have a part to play. </p>
<p>The <a href="https://www.scienceathome.org/games/corona-minister-game/about-corona-minister/">Corona Minister</a> game allows people to explore the consequences of different policy interventions as they navigate trade-offs between public health, the economy and civil rights. Elsewhere, researchers in Denmark have created a <a href="https://www.reuters.com/business/healthcare-pharmaceuticals/denmark-trial-uses-virtual-reality-game-boost-covid-vaccinations-2021-06-07/">VR gaming experience</a> where citizens navigate through crowded scenes and try to avoid infection. The aim of the experience is to help participants engage with the complexity of disease spread and the role played by vaccination. </p>
<figure>
<iframe width="440" height="260" src="https://www.youtube.com/embed/ZHnZDxfewX8?wmode=transparent&start=0" frameborder="0" allowfullscreen=""></iframe>
</figure>
<p>Making progress in how we can effectively think, decide and act together is an area that receives almost no research investment. We think <a href="https://www.nesta.org.uk/report/future-minds-and-machines/">using AI</a> to make the most of the distributed collective intelligence of large, diverse groups is a major frontier for innovation, and a huge opportunity to prepare the population for a future crisis.</p>
<h2>Invest in bottom-up initiatives</h2>
<p>From Ebola to COVID, we’ve learned time and again that crises require both top-down and bottom-up responses. So while the Global Pandemic Radar is a great step forward, governments who are serious about crisis prevention and response need to start supporting the digital and social infrastructures that enable communities to act intelligently themselves. </p>
<p>In 2020, we saw how existing systems of community action were able to pivot quickly to focus on COVID-19. One of them was <a href="http://metasub.org/">MetaSUB</a>, a global project to build microbial portraits of urban transit systems that’s been around since 2015. With a network of volunteers and scientists in over 100 cities, they take regular swabs from trains and escalators, testing the pathogens they find for any markers of antibiotic resistance.</p>
<p>The pandemic saw them quickly set up the <a href="http://metasub.org/projects/#">MetaCOV project</a>, applying their previous methodology to see how microbial samples changed during the pandemic. Their data <a href="https://spectrum.ieee.org/the-human-os/biomedical/devices/swabbing-the-worlds-cities-for-coronavirus">helped show</a> that the longer COVID-19 was on a surface, the less likely it was to make someone sick. </p>
<figure>
<iframe width="440" height="260" src="https://www.youtube.com/embed/EM0qE1NkYao?wmode=transparent&start=0" frameborder="0" allowfullscreen=""></iframe>
</figure>
<p>Then there’s the <a href="https://www.cmu.edu/news/stories/archives/2017/september/flu-forecasts.html">FluCast forecasting system</a>, which has been tapping into the “<a href="https://www.amazon.co.uk/Wisdom-Crowds-Many-Smarter-Than/dp/0349116059">wisdom of crowds</a>” to predict seasonal flu trends for the US Centres for Disease Control since 2015. The system was swiftly repurposed into <a href="https://delphi.cmu.edu/covidcast/">CovidCast</a> in 2020, which relies on open data sources and the participation of volunteers. CovidCast now offers real-time data across a range of indicators – including mask wearing and COVID-related visits to doctors – to forecast regional spikes in COVID infections and hospitalisations.</p>
<p>That these systems were already present and connected meant they could rapidly be deployed to serve pressing new requirements. Many community-led responses have, of course, emerged to play a vital role without any existing institutional support, such as groups <a href="https://frontline.live/">creating PPE</a> for struggling hospitals, and communities in India and Nepal tracking <a href="https://restofworld.org/2021/pandemic-desperate-indians-turning-online-crowdsourcing-for-help/?mc_cid=4cd2e241a6&mc_eid=ccb32e3378">oxygen supplies</a> and <a href="https://hospitalbedsnepal.com/">hospital bed availability</a>. Many of these new groups should be supported so that they can mobilise swiftly in future emergencies.</p>
<p>On top of that, greater proactive investment, following the lead of organisations like <a href="https://omidyar.com/the-community-infrastructure-fund-for-mutual-aid/">the Omidyar Network</a>, should now be directed towards community infrastructure. And government institutions should acknowledge that it’s currently too difficult for community projects to <a href="https://www.wilsoncenter.org/publication/stitching-together-solution-lessons-open-source-hardware-response-covid-19">connect into institutions</a>. If they’re <a href="https://journals.sagepub.com/doi/pdf/10.1177/0956247817721413">excluded from</a> formal planning, such groups can’t offer their collective intelligence for the collective good. </p>
<h2>Harnessing collective intelligence</h2>
<p>At its best, collective intelligence can help us respond to crises with greater confidence, clarity and cooperation. But we need to start building and reinforcing these schemes and systems now – before the next crisis.</p>
<p>The pandemic has been tough. But it has also thrust our collective intelligence under the spotlight, whether through neighbourhood WhatsApp groups or international scientific research. As we move towards COVID recovery, placing our bets on new initiatives like the Global Pandemic Radar, we must ensure these lessons aren’t forgotten. We must now invest in the combined power of data, technology and people, which will help us avoid the next outbreak and counter society’s next big crisis.</p>
<hr>
<p><em>This article is part of a series on recovering from the pandemic in a way that makes societies more resilient and able to deal with future challenges. It is supported by <a href="https://theconversation.com/uk/topics/resilience-in-recovery-106634">PreventionWeb</a>, a platform from the UN Office for Disaster Risk Reduction. Read more coverage <a href="https://theconversation.com/uk/topics/resilient-recovery-series-106366">here</a>.</em></p><img src="https://counter.theconversation.com/content/162721/count.gif" alt="The Conversation" width="1" height="1" />
<p class="fine-print"><em><span>Aleks Berditchevskaia receives funding from the United Nations Development Programme, the Economic and Social Research Council and the UK Humanitarian Innovation Hub. </span></em></p><p class="fine-print"><em><span>Kathy Peach receives funding from the UK Humanitarian Innovation Hub, the United Nations Development Programme, the Omidyar Network, Wellcome and the Patrick J. McGovern Foundation. She is an adviser to the World Economic Forum. Kathy is a Labour party member. </span></em></p>The WHO is creating a Global Pandemic Radar – an example of collective intelligence that must learn lessons from this pandemic.Aleks Berditchevskaia, Principal Researcher, Centre for Collective Intelligence Design, NestaKathy Peach, Director of the Centre for Collective Intelligence Design, NestaLicensed as Creative Commons – attribution, no derivatives.tag:theconversation.com,2011:article/1373112020-04-30T11:07:20Z2020-04-30T11:07:20ZStrong opinions are irrational – here’s why we should all be agnostic<figure><img src="https://images.theconversation.com/files/331394/original/file-20200429-51461-1epefk0.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=496&fit=clip" /><figcaption><span class="caption">Trump's supporters appreciate his strong opinons.</span> <span class="attribution"><a class="source" href="https://www.shutterstock.com/image-photo/tampa-florida-july-31-2018-president-1147142267">jctabb/Shutterstock</a></span></figcaption></figure><p>Suppose you are on a trial jury trying to decide whether the defendant is guilty. You are discussing the case with your fellow jurors who you know have exactly the same evidence as you, and are just as good at assessing the evidence. You think the defendant is guilty, while your peers think he is innocent. After lengthy discussion, you still disagree. What is the rational response to this disagreement? Is there a logical way out of such an impasse? </p>
<p>This is a common situation, but it is deeply puzzling. To answer the question of what the rational response to disagreements is, we should distinguish the <em>psychological</em> question of what people do from the <em>philosophical</em> question of what people ought to do.</p>
<p>The issues with what people do <a href="https://www.nature.com/articles/s41393-020-0446-2">are well known</a>. Benjamin Franklin, one of the founding fathers of the US, <a href="http://www.benjamin-franklin-history.org/constitutional-convention/">wrote that</a>: “Most men … think themselves in possession of all truth, and that wherever others differ from them, it is so far error.” </p>
<p>This has been backed up by research. Most people <a href="https://doi.org/10.1038/s41393-020-0446">ignore evidence that would contradict their beliefs</a>, regardless of whether they are right. They are even biased in their detection of bias – they find it in other people, <a href="https://psyarxiv.com/d38rt/">but not in themselves</a>.</p>
<p>It is when we consider what we <em>should</em> do that we realise such responses are irrational – they are often <a href="https://theconversation.com/feelings-whats-the-point-of-rational-thought-if-emotions-always-take-over-128592">based on emotions rather than logic</a>. Those who hold that their opinions are right and everyone else is mistaken are guilty of being arbitrary. They have most likely acted impulsively, failing to make a rational assessment of the argument. </p>
<p>To avoid being arbitrary, you should be humble and conciliate: move your opinions towards the other person’s. Similarly, they should move their opinion towards yours and you should both become agnostic. We are talking here about disagreement between peers, people who are equally intelligent. This is nevertheless extremely counter-intuitive for most of us.</p>
<p>But conciliating has some successes. The “<a href="https://theconversation.com/how-to-unleash-the-wisdom-of-crowds-52774">wisdom of the crowd</a>” is the well-known phenomenon that groups <a href="http://www.penguinrandomhouse.com/books/175380/the-wisdom-of-crowds-by-james-surowiecki/">can produce very accurate opinions</a>. This can be <a href="https://www.loebclassics.com/view/aristotle-politics/1932/pb_LCL264.635.xml">traced to the ancient philosopher Aristotle</a> and was popularised when the English scientist Francis Galton noticed that <a href="https://www.all-about-psychology.com/support-files/the-wisdom-of-crowds.pdf">the average of 800 guesses</a> of the weight of a bull was within 1% of the correct weight.</p>
<figure class="align-center ">
<img alt="" src="https://images.theconversation.com/files/331405/original/file-20200429-51513-1y8kn0g.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=754&fit=clip" srcset="https://images.theconversation.com/files/331405/original/file-20200429-51513-1y8kn0g.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=600&h=399&fit=crop&dpr=1 600w, https://images.theconversation.com/files/331405/original/file-20200429-51513-1y8kn0g.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=600&h=399&fit=crop&dpr=2 1200w, https://images.theconversation.com/files/331405/original/file-20200429-51513-1y8kn0g.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=600&h=399&fit=crop&dpr=3 1800w, https://images.theconversation.com/files/331405/original/file-20200429-51513-1y8kn0g.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=754&h=501&fit=crop&dpr=1 754w, https://images.theconversation.com/files/331405/original/file-20200429-51513-1y8kn0g.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=754&h=501&fit=crop&dpr=2 1508w, https://images.theconversation.com/files/331405/original/file-20200429-51513-1y8kn0g.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=754&h=501&fit=crop&dpr=3 2262w" sizes="(min-width: 1466px) 754px, (max-width: 599px) 100vw, (min-width: 600px) 600px, 237px">
<figcaption>
<span class="caption">Should you stand up for your beliefs?</span>
<span class="attribution"><a class="source" href="https://www.shutterstock.com/image-photo/multi-ethnic-jurors-witness-stand-court-121503349">sirtravelalot/Shutterstock</a></span>
</figcaption>
</figure>
<p>But this commonsense view has some disturbing results if we follow it through, as it becomes impossible to maintain any opinion in the light of peers who disagree with you. When faced with a disagreeing juror, you should give up your belief. When it comes to 11 other jurors disagreeing with your view that a defendant is guilty, it is more likely that you made a mistake than that all the others did, so you should change your mind and conclude that the defendant is innocent. </p>
<p>Almost any controversial opinion becomes irrational. You probably have at least one strong political opinion on which intelligent people disagree with you. According to <a href="https://www.merriam-webster.com/dictionary/conciliationism">conciliationism</a>, this is irrational. The only rational position becomes a radical agnosticism, refraining from any strong opinions that are not shared by most other people. It is irrational to disagree with the crowd. </p>
<p>But philosopher <a href="http://www.princeton.edu/%7Eadame/">Adam Elga</a> <a href="https://philpapers.org/rec/ELGHTD">sees this as “spineless”</a> and argues that you don’t always have to agree with the crowd to be agnostic. Consider people who have radically different political views to yours. These radically different political views must be based on a radically different worldview. If you think that this worldview is radically wrong, perhaps you should decide that they are not your peers after all and discount their opinion.</p>
<p>But I argue that this gets things backwards. If you initially thought they are as intelligent and knowledgeable as you, you should take their views to be evidence that your entire worldview is wrong. There are limits of course. We should only worry about the beliefs of those who are at least as well-informed and as good at assessing the evidence as we are. Still, many people find this approach uncomfortable.</p>
<p>But what benefits could radical agnosticism have in society? Politicians often have to make decisions in the light of experts disagreeing, as we are now seeing in responses to the coronavirus. When there is a choice between incompatible paths, it might be best to take one despite having low confidence that it is the right path to take – opting for the one that seems to be the best. In such cases, however, new evidence, which might still be inconclusive, could show that the best course is to change from one path to another. So making a U-turn, which is politically embarrassing, is actually more rational than sticking with a faulty initial approach.</p>
<p>When it comes to science, revolutions have come from people who completely disagreed with their peers. Scientists might rationally work on something which they and others think will fail, as the benefits of being right make it worthwhile, despite the low probability of success. But if they genuinely believe they are right, they should be able to convince others. Einstein discovered a revolutionary and deeply counter-intuitive theory of gravity, but it did not take very long for other scientists to adopt it.</p>
<p>Strangely, those with strong beliefs tend to be admired. The human mind hates uncertainty, so it is comforting to be told what to think, and to form settled opinions. But it is not rational. As the philosopher <a href="https://plato.stanford.edu/entries/russell/">Bertrand Russell</a> <a href="https://russell-j.com/0583TS.HTM">wrote</a>: “The fundamental cause of the trouble is that in the modern world the stupid are cocksure while the intelligent are full of doubt.” </p>
<p>It’s an insightful comment that we should all ponder. Whether we are able to fully embrace radical agnosticism or not, chances are the world would be a better place if we started questioning our own beliefs a bit more.</p><img src="https://counter.theconversation.com/content/137311/count.gif" alt="The Conversation" width="1" height="1" />
<p class="fine-print"><em><span>Darren Bradley receives funding from the Arts and Humanities Research Council.</span></em></p>Those with strong beliefs tend to be admired.Darren Bradley, Associate Professor, University of LeedsLicensed as Creative Commons – attribution, no derivatives.tag:theconversation.com,2011:article/1365472020-04-28T10:56:43Z2020-04-28T10:56:43ZHow digital tools can keep democracy thriving during lockdown<figure><img src="https://images.theconversation.com/files/330762/original/file-20200427-145499-kqbz6s.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=496&fit=clip" /><figcaption><span class="caption">Going virtual.</span> <span class="attribution"><a class="source" href="https://www.shutterstock.com/image-photo/big-ben-houses-parliament-sunset-illustration-788457124">Shutterstock</a></span></figcaption></figure><p>This is not business as usual for democracy. As the COVID-19 pandemic rages, governments at all levels are having to work remotely, and <a href="https://www.idea.int/news-media/multimedia-reports/global-overview-covid-19-impact-elections">postpone elections</a> and parliamentary sessions. To further complicate things, they are having to do this while delivering fast-paced and effective decision-making. </p>
<p>But democracy must go on. Thankfully, digital tools can help keep parliaments and governments thriving in a way that enhances rather than threatens democracy. In fact, the lockdown is a key opportunity to experiment with digital methods to support democratic institutions and citizen engagement into the future.</p>
<p>Digital tools certainly can support remote decision-making. <a href="https://www.nesta.org.uk/report/digital-democracy-the-tools-transforming-political-engagement/">Our research</a> into digital democracy shows how such tools can be used to harness the <a href="https://theconversation.com/brexit-wisdom-of-crowds-proves-effective-predictor-of-britains-chaotic-eu-departure-119906">wisdom of the crowd</a> at different stages in the democratic process.</p>
<figure class="align-center zoomable">
<a href="https://images.theconversation.com/files/330737/original/file-20200427-145513-zxq5ek.png?ixlib=rb-1.1.0&q=45&auto=format&w=1000&fit=clip"><img alt="" src="https://images.theconversation.com/files/330737/original/file-20200427-145513-zxq5ek.png?ixlib=rb-1.1.0&q=45&auto=format&w=754&fit=clip" srcset="https://images.theconversation.com/files/330737/original/file-20200427-145513-zxq5ek.png?ixlib=rb-1.1.0&q=45&auto=format&w=600&h=355&fit=crop&dpr=1 600w, https://images.theconversation.com/files/330737/original/file-20200427-145513-zxq5ek.png?ixlib=rb-1.1.0&q=30&auto=format&w=600&h=355&fit=crop&dpr=2 1200w, https://images.theconversation.com/files/330737/original/file-20200427-145513-zxq5ek.png?ixlib=rb-1.1.0&q=15&auto=format&w=600&h=355&fit=crop&dpr=3 1800w, https://images.theconversation.com/files/330737/original/file-20200427-145513-zxq5ek.png?ixlib=rb-1.1.0&q=45&auto=format&w=754&h=446&fit=crop&dpr=1 754w, https://images.theconversation.com/files/330737/original/file-20200427-145513-zxq5ek.png?ixlib=rb-1.1.0&q=30&auto=format&w=754&h=446&fit=crop&dpr=2 1508w, https://images.theconversation.com/files/330737/original/file-20200427-145513-zxq5ek.png?ixlib=rb-1.1.0&q=15&auto=format&w=754&h=446&fit=crop&dpr=3 2262w" sizes="(min-width: 1466px) 754px, (max-width: 599px) 100vw, (min-width: 600px) 600px, 237px"></a>
<figcaption>
<span class="caption">A typology of digital democracy, from p13 ‘Digital Democracy: The Tools Transforming Political Engagement’, Nesta (2017).</span>
<span class="attribution"><span class="source">Nesta</span>, <span class="license">Author provided</span></span>
</figcaption>
</figure>
<p>At a personal level, whether you’re a politician, a policymaker, or just an interested citizen, digital tools can efficiently collect information from different sources to provide an overview of the options and people’s views on them.</p>
<p>To weigh up the pros and cons of a particular policy or idea, for example, platforms such as <a href="https://yrpri.org/domain/3">Your Priorities</a> and <a href="http://consulproject.org/en/">Consul</a> allow people to contribute to the debate. Meanwhile, to quickly gauge levels of support for different options, platforms such as <a href="https://www.allourideas.org/">All Our Ideas</a> enable users to rank a live bank of ideas. And if you need to source questions from – or gauge the needs of – citizens, head to platforms such as <a href="https://www.sli.do/">Sli.do</a>, or use task management tools like <a href="https://trello.com/">Trello</a> and <a href="https://asana.com/">Asana</a>.</p>
<p>For those thinking of engaging in this way, however, it is important to have a clear idea of <a href="https://www.nesta.org.uk/report/digital-democracy-the-tools-transforming-political-engagement/">what you want</a>. Pick the right tools to suit your goals. Don’t engage for engagement’s sake. Be honest with those you’re engaging with: what’s involved and what are you going to do with the input? And don’t think the results will be a definitive quick fix: you’ll still need to devote real world time and thought to exploring the options. </p>
<h2>Maintaining democracy during lockdown</h2>
<p>But this is also about the big democratic institutions. While pressure to take crucial decisions quickly and remotely can put strain on the core principles of democracy, many parliaments have been exploring ways to move online while maintaining scrutiny and participation.</p>
<p>The UK parliament has successfully made the transition to hold its first <a href="https://www.parliament.uk/business/news/2020/april1/hybrid-house-of-commons/">prime minister’s questions online</a>; the <a href="https://www.europarl.europa.eu/news/en/headlines/eu-affairs/20200408STO76807/how-parliament-works-during-a-pandemic">EU parliament</a> has moved to e-voting; the <a href="https://www.parliament.scot/newsandmediacentre/114963.aspx">Scottish parliament</a> is holding a virtual Question Time; and <a href="https://www.parliament.nz/en/get-involved/features/faq-parliament-during-covid-19-alert-level-4/">New Zealand’s</a> select committees are going online, too. Most of these arrangements seek to recreate physical meetings via video call, but the creative use of other tools could provide even more effective results.</p>
<p>The public can be directly involved in the decision-making process, too. Initiatives such as <a href="https://www.nesta.org.uk/feature/six-pioneers-digital-democracy/vtaiwan/">vTaiwan</a>, France’s <a href="https://www.nesta.org.uk/feature/six-pioneers-digital-democracy/parlement-et-citoyens/">Parlement et citoyens</a>, and <a href="https://www.nesta.org.uk/feature/six-pioneers-digital-democracy/decide-madrid/">Decide Madrid</a>, for example, enable public participation in important government discussions around law-making, budgets and other issues, potentially tailoring decisions more closely to citizen’s needs and creating a greater degree of consensus. Further inspiration can be found from grassroots and <a href="https://www.nesta.org.uk/blog/meet-19-pioneers-shaking-democracy/">civil society democratic innovators</a> who have been testing and developing many of these approaches.</p>
<figure class="align-center ">
<img alt="" src="https://images.theconversation.com/files/330761/original/file-20200427-145503-8lfxkc.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=754&fit=clip" srcset="https://images.theconversation.com/files/330761/original/file-20200427-145503-8lfxkc.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=600&h=338&fit=crop&dpr=1 600w, https://images.theconversation.com/files/330761/original/file-20200427-145503-8lfxkc.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=600&h=338&fit=crop&dpr=2 1200w, https://images.theconversation.com/files/330761/original/file-20200427-145503-8lfxkc.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=600&h=338&fit=crop&dpr=3 1800w, https://images.theconversation.com/files/330761/original/file-20200427-145503-8lfxkc.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=754&h=425&fit=crop&dpr=1 754w, https://images.theconversation.com/files/330761/original/file-20200427-145503-8lfxkc.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=754&h=425&fit=crop&dpr=2 1508w, https://images.theconversation.com/files/330761/original/file-20200427-145503-8lfxkc.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=754&h=425&fit=crop&dpr=3 2262w" sizes="(min-width: 1466px) 754px, (max-width: 599px) 100vw, (min-width: 600px) 600px, 237px">
<figcaption>
<span class="caption">Press here…</span>
<span class="attribution"><a class="source" href="https://www.shutterstock.com/image-illustration/indonesia-high-resolution-vote-concept-202352587">Shutterstock</a></span>
</figcaption>
</figure>
<p>Introducing digital tools could also address several existing <a href="https://www.nesta.org.uk/report/smarter-select-committees/">“pain points”</a>, such as simplifying the submission procedure for formal evidence to parliamentary committees. And democracy could be further supported by fact-checking initiatives aimed at <a href="https://www.nesta.org.uk/blog/fake-news-what-is-it-and-how-can-we-tackle-it/">avoiding misinformation and reducing panic</a>.</p>
<h2>Socially-distanced elections</h2>
<p>Since the start of the COVID-19 pandemic, elections and referendums have been postponed, or have required physical distancing or postal voting policies to go ahead. </p>
<p>But plenty of systems for online voting already exist. In 2005, Estonia became the first country to host nationwide <a href="https://e-estonia.com/solutions/e-governance/i-voting/">online elections</a>; <a href="https://www.ch.ch/en/demokratie/voting-online/what-is-e-voting/">Switzerland’s cantons</a> have offered e-voting; New Zealand has an <a href="https://vote.nz/elections-and-more/all-events/2020/2020-general-election/covid-19-and-the-2020-general-election/">existing system</a> to enable overseas citizens to vote which could be extended; India has <a href="https://eci.gov.in/evm/">electronic voting</a> in polling stations; and online voting has been trialled for <a href="https://inews.co.uk/news/politics/vote-online-uk-can-i-general-election-2019-digital-voting-1339282">local elections</a> in the UK and <a href="https://theconversation.com/protecting-online-elections-in-canada-105764">Canada</a>.</p>
<p>These initiatives could now potentially provide the basis for digital voting during the current and future crises.</p>
<p>Indeed, by allowing civic participation at various points in the decision-making process, governments could also ease citizens’ anxiety by giving them a greater sense of control over the circumstances affecting their lives. Citizens with particular skills could also be approached to contribute digitally on issues relevant to their expertise, while existing deliberation, <a href="https://www.climateassembly.uk/news/climate-assembly-uk-complete-its-work-path-net-zero-through-online-video-conferencing/">such as the Climate Assembly UK</a>, could be moved online.</p>
<p>Any plans to move this way must <a href="http://documents.worldbank.org/curated/en/907721570027981778/Citizen-Engagement-Emerging-Digital-Technologies-Create-New-Risks-and-Value">consider accessibility, diversity and inclusion</a>, but such initiatives could lay the groundwork for the future of democracy. </p>
<h2>What next?</h2>
<p>Coronavirus has forced a rapid move towards online methods of decision-making – accelerating a process of adoption which was previously predicted to take years rather than days. But this period should be seen as a <a href="https://www.nesta.org.uk/feature/innovation-methods/innovation-prototyping/">rapid prototyping phase</a> of digital democracy on a large scale. We can now test and adapt innovative tools and methods without the usual levels of risk aversion acting as a barrier. </p>
<p>The tools chosen, and the way in which they are used, will have significant ramifications for the use of digital methods in the long term. Coming out of lockdown, changes to our ways of working – and thinking – will be inevitable. But <a href="https://www.hansardsociety.org.uk/publications/proposals-for-a-virtual-parliament">what we learn</a> from this time will inform decisions about the use of digital methods as democracies tackle some of the biggest economic, social and health challenges of our time.</p><img src="https://counter.theconversation.com/content/136547/count.gif" alt="The Conversation" width="1" height="1" />
<p class="fine-print"><em><span>Nesta has previously received funding from the European Union's Horizon 2020 Programme for the DECODE project.
Rosalyn volunteers as a board member of the European Youth Forum.</span></em></p>Innovative ideas during the lockdown could protect democracy and make it stronger and more inclusive.Rosalyn Old, Researcher, Government Innovation, NestaLicensed as Creative Commons – attribution, no derivatives.tag:theconversation.com,2011:article/1280992019-12-04T16:56:20Z2019-12-04T16:56:20ZUK election 2019: what the ‘wisdom of crowds’ forecasts<figure><img src="https://images.theconversation.com/files/304556/original/file-20191201-156073-1r03je9.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=496&fit=clip" /><figcaption><span class="caption">And the crowd says...</span> <span class="attribution"><span class="source">Shutterstock</span></span></figcaption></figure><p>Forecasting outcomes in 21st-century politics can sometimes feel like a fool’s undertaking. The media landscape is crowded with conflicting interpretations and analyses of campaigns and voting intentions – and the failure of many polls to forecast the outcome of the <a href="http://eprints.ncrm.ac.uk/3789/1/Report_final_revised.pdf">2015 UK general election</a> in particular has led to each new poll’s accuracy being <a href="https://www.theguardian.com/commentisfree/2019/nov/12/polls-2019-election">hotly debated</a> in a way that was unthinkable a decade ago.</p>
<p>So how else can we cut through the noise to build a more objective assessment of what is likely to happen on December 12? We argue that it’s worth considering crowd forecasting as an alternative to more traditional methods. Why? Because the crowd has proved to be right about Brexit so far.</p>
<p>Earlier this year, we wrote about the ability of an <a href="https://theconversation.com/brexit-wisdom-of-crowds-proves-effective-predictor-of-britains-chaotic-eu-departure-119906">amateur, global crowd to accurately forecast</a> the extension of Article 50 in March and the vote share gained by Change UK and the Brexit Party in the May 2019 European parliament elections. In retrospect, these outcomes may seem obvious and inevitable, but at the time, uncertainty was rife.</p>
<hr>
<p>
<em>
<strong>
Read more:
<a href="https://theconversation.com/brexit-wisdom-of-crowds-proves-effective-predictor-of-britains-chaotic-eu-departure-119906">Brexit: wisdom of crowds proves effective predictor of Britain's chaotic EU departure</a>
</strong>
</em>
</p>
<hr>
<p>At Nesta’s Centre for Collective Intelligence Design, we’ve been carrying out a <a href="https://www.nesta.org.uk/project/crowd-predictions/">year-long experiment</a> to put crowd wisdom to the test in the face of the high uncertainty caused by Brexit. We’ve been working on this with BBC Future and the online forecasting platform Good Judgment Open, which first showed that a crowd of non-specialists could make surprisingly <a href="http://journal.sjdm.org/17/17408/jdm17408.pdf">accurate estimates of probability</a> about geopolitical events.</p>
<p>When it comes to forecasting unexpected election outcomes, there is some precedent. Although Good Judgment forecasters didn’t anticipate a Donald Trump victory outright, they were the <a href="https://www.washingtonpost.com/news/monkey-cage/wp/2016/11/30/which-election-forecast-was-the-most-accurate-or-rather-the-least-wrong/">least wrong compared to other prediction methods</a>. Indeed, crowd-based prediction markets have even been <a href="http://journal.sjdm.org/17/17815/jdm17815.pdf">shown to outperform intelligence analysts</a>. </p>
<p>Since the summer, our crowd has continued to perform strongly on Brexit questions. For example, it correctly decided an October Article 50 extension was the most likely outcome from a range of options (which included leaving without a deal and leaving with a new deal), and accurately forecast what the value of the pound versus the Euro would be on November 1, 2019.</p>
<p>We’re now monitoring their evolving forecast for what will happen in the UK general election.</p>
<h2>What is crowd forecasting?</h2>
<p>The process is based on the “<a href="https://theconversation.com/brexit-wisdom-of-crowds-proves-effective-predictor-of-britains-chaotic-eu-departure-119906">wisdom of crowds</a>” principle. When large groups of distributed individuals are all asked to make estimates on numerical problems, they rely on unique personal skills and sources of information. They also tend to make errors in different ways – for example, some may have a tendency to overestimate while others do the opposite. This means that when you aggregate and average across all of the estimates, their errors cancel out, making them more accurate than individuals on most occasions.</p>
<p>But that’s not all. Our crowd forecasting also promotes the open sharing of information between participants and <a href="https://journals.sagepub.com/doi/10.1177/0956797614524255">allows the adjustment of estimates over time</a>. This makes it different from other popular approaches to predicting election outcomes such as opinion polling and prediction markets.</p>
<figure class="align-center ">
<img alt="" src="https://images.theconversation.com/files/305043/original/file-20191203-67023-7gukmo.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=754&fit=clip" srcset="https://images.theconversation.com/files/305043/original/file-20191203-67023-7gukmo.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=600&h=400&fit=crop&dpr=1 600w, https://images.theconversation.com/files/305043/original/file-20191203-67023-7gukmo.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=600&h=400&fit=crop&dpr=2 1200w, https://images.theconversation.com/files/305043/original/file-20191203-67023-7gukmo.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=600&h=400&fit=crop&dpr=3 1800w, https://images.theconversation.com/files/305043/original/file-20191203-67023-7gukmo.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=754&h=503&fit=crop&dpr=1 754w, https://images.theconversation.com/files/305043/original/file-20191203-67023-7gukmo.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=754&h=503&fit=crop&dpr=2 1508w, https://images.theconversation.com/files/305043/original/file-20191203-67023-7gukmo.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=754&h=503&fit=crop&dpr=3 2262w" sizes="(min-width: 1466px) 754px, (max-width: 599px) 100vw, (min-width: 600px) 600px, 237px">
<figcaption>
<span class="caption">‘They said that?’</span>
<span class="attribution"><a class="source" href="https://www.shutterstock.com/image-photo/london-united-kingdom-20th-september-2019-1510887680">Shutterstock</a></span>
</figcaption>
</figure>
<p>Participants are incentivised to update their forecasts as new information becomes available, in order to improve their personal “<a href="https://www.met-learning.eu/pluginfile.php/5277/mod_resource/content/6/www/english/msg/ver_prob_forec/uos2/uos2_ko1.htm">Brier score</a>” (which measures the accuracy of their performance). It’s been shown that this approach helps to promote <a href="https://www.researchgate.net/publication/328617335_Forecasting_tournaments_epistemic_humility_and_attitude_depolarization">a more objective assessment of evidence</a> from different sides of the political spectrum, in contrast to the reported polarising effects of social media bubbles.</p>
<h2>Forecasting the election</h2>
<p>Most recently, we’ve been asking our forecasters to estimate the likelihood of a single party majority in the House of Commons. In the first half of November, we saw the option of a “Conservative majority” battle it out with “No overall majority”. But this changed from November 14 when the crowd first assigned the decisive highest likelihood to the Conservatives having a majority.</p>
<p>Since then, the consensus estimate has consolidated. It currently assigns a 75% likelihood to a Conservative majority, while the likelihood of a Labour majority is now 0%. The likelihood of a hung parliament is at 25%.</p>
<p>We’ve also asked our crowd to predict how many of ten key marginal seats will change hands. Currently they’re forecasting that five or six of the ten will have new parties representing them.</p>
<figure class="align-center ">
<img alt="" src="https://images.theconversation.com/files/305191/original/file-20191204-70144-1mxss0r.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=754&fit=clip" srcset="https://images.theconversation.com/files/305191/original/file-20191204-70144-1mxss0r.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=600&h=410&fit=crop&dpr=1 600w, https://images.theconversation.com/files/305191/original/file-20191204-70144-1mxss0r.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=600&h=410&fit=crop&dpr=2 1200w, https://images.theconversation.com/files/305191/original/file-20191204-70144-1mxss0r.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=600&h=410&fit=crop&dpr=3 1800w, https://images.theconversation.com/files/305191/original/file-20191204-70144-1mxss0r.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=754&h=515&fit=crop&dpr=1 754w, https://images.theconversation.com/files/305191/original/file-20191204-70144-1mxss0r.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=754&h=515&fit=crop&dpr=2 1508w, https://images.theconversation.com/files/305191/original/file-20191204-70144-1mxss0r.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=754&h=515&fit=crop&dpr=3 2262w" sizes="(min-width: 1466px) 754px, (max-width: 599px) 100vw, (min-width: 600px) 600px, 237px">
<figcaption>
<span class="caption">What the crowd has forecast over time.</span>
<span class="attribution"><span class="source">Nesta.</span>, <span class="license">Author provided</span></span>
</figcaption>
</figure>
<p>Elsewhere, the last week has seen <a href="https://metro.co.uk/2019/12/02/tory-lead-labour-narrows-just-9-points-latest-election-poll-11254616/">reports of Labour closing the polling gap</a> and record numbers of young voters registering in key marginal seats. Prediction <a href="https://smarkets.com/event/829238/politics/uk/next-uk-general-election/overall-ge-majority">markets have seen traders adjusting</a> to these media reports with a slight uptick in the likelihood of “no overall majority” over the last week.</p>
<p>Noted political commentator Sir John Curtice has also warned that recent changes to the balance between <a href="https://www.telegraph.co.uk/politics/2019/11/30/boris-johnsons-dreams-majority-could-hinge-badly-remain-vote/">Liberal Democrat and Labour support</a> may affect the Conservatives’ chances of a majority. This contrasts with the prediction issued by <a href="https://yougov.co.uk/uk-general-election-2019/">YouGov’s much lauded MRP model</a> that has estimated a 60+ seat majority for Conservatives.</p>
<p>So will the crowd manage to cut through the Brexit noise once again? We’ll see in seven days. In the meantime, you can register your own forecast by signing up on the <a href="https://goodjudgment.io/nesta/register.html">Good Judgment</a> platform.</p><img src="https://counter.theconversation.com/content/128099/count.gif" alt="The Conversation" width="1" height="1" />
<p class="fine-print"><em><span>Kathy Peach is a member of the Labour party. </span></em></p><p class="fine-print"><em><span>Aleks Berditchevskaia does not work for, consult, own shares in or receive funding from any company or organisation that would benefit from this article, and has disclosed no relevant affiliations beyond their academic appointment.</span></em></p>Crowd-based prediction markets have even been shown to outperform intelligence analysts.Aleks Berditchevskaia, Senior Researcher, Centre for Collective Intelligence Design, NestaKathy Peach, Head of the Centre for Collective Intelligence Design, NestaLicensed as Creative Commons – attribution, no derivatives.tag:theconversation.com,2011:article/1199062019-07-17T10:16:50Z2019-07-17T10:16:50ZBrexit: wisdom of crowds proves effective predictor of Britain’s chaotic EU departure<figure><img src="https://images.theconversation.com/files/283285/original/file-20190709-44457-1yl0aze.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=496&fit=clip" /><figcaption><span class="caption">
</span> <span class="attribution"><a class="source" href="https://www.shutterstock.com/download/success?u=http%3A%2F%2Fdownload.shutterstock.com%2Fgatekeeper%2FW3siZSI6MTU2MjY5NTA1MiwiYyI6Il9waG90b19zZXNzaW9uX2lkIiwiZGMiOiJpZGxfMTE2NjYwMDA4NiIsImsiOiJwaG90by8xMTY2NjAwMDg2L21lZGl1bS5qcGciLCJtIjoxLCJkIjoic2h1dHRlcnN0b2NrLW1lZGlhIn0sIlF1d2VPanVWMFZ0T05iTmZNN29IY2J6ak1MdyJd%2Fshutterstock_1166600086.jpg&pi=33421636&m=1166600086&src=qfIN1MCgipeTEwimblNvTw-2-14">Shutterstock</a></span></figcaption></figure><p>Winston Churchill <a href="http://www.churchill-society-london.org.uk/RusnEnig.html">once described Russia</a> as “a riddle wrapped in a mystery inside an enigma”. Many feel the same about Brexit.</p>
<p>Achieving Brexit is a fiendishly complex task. And when the process is led by a government with a tiny majority, blocked by a divided parliament and confronted with a split country, the path out of the EU looks far from certain. With more potential endings than a <a href="https://en.wikipedia.org/wiki/Black_Mirror:_Bandersnatch">Bandersnatch</a>-style plot line, it’s no wonder that the one thing commentators and experts seem to agree on is that <a href="https://inews.co.uk/opinion/brexit-no-deal-second-referendum-impossible/">Brexit is unpredictable</a>.</p>
<p>In these times of <a href="https://qbeeurope.com/unpredictability/coping-with-an-unpredictable-world/">radical uncertainty</a>, accurately forecasting what will happen becomes more difficult – as statistical models built on historical data often don’t work. It also becomes more necessary as individuals and organisations attempt to navigate their way through the unknown. Anyone who <a href="https://www.itv.com/news/2019-05-01/no-deal-brexit-ferry-contracts-to-be-scrapped/">chartered ferries</a> or <a href="https://www.bbc.co.uk/news/world-47640908">stockpiled toilet roll</a> before March 31, when the UK was originally scheduled to leave, could attest to this.</p>
<p>So what to do when statistical models can’t help? </p>
<h2>Ask the crowd</h2>
<p>Crowd forecasting is a relatively new approach to predicting the future. It’s getting serious attention, because as the world becomes more complex and uncertain, it’s unlikely that any single person will have enough information to build a complete picture.</p>
<p>When individuals make a prediction, partial information and personal experience can lead to errors. These individual errors, however, tend to be cancelled out when predictions from a group of people <a href="https://www.cambridge.org/core/books/collective-wisdom/some-microfoundations-of-collective-wisdom/29B1A9227DD7D3F531840705AAD0CABF">are aggregated</a>. Companies, such as the car manufacturer Ford, have cottoned on and have used one type of crowd forecasting called a <a href="https://en.wikipedia.org/wiki/Prediction_market">Prediction Market</a> to forecast vehicle sales. This has been found to be <a href="https://www.researchgate.net/publication/266658533_Corporate_Prediction_Markets_Evidence_from_Google_Ford_and_Firm_X1">more accurate</a> than traditional forecasting methods.</p>
<p>This harnessing of collective human intelligence is sometimes called <a href="https://www.amazon.co.uk/Wisdom-Crowds-Many-Smarter-Than/dp/0349116059">the “wisdom of crowds”</a>, a term popularised by James Surowiecki. He argued that when a diverse group of people come up with an answer it is likely to be better than that of the smartest person in that group, or even a group of experts. The accuracy of the crowd has been demonstrated through examples ranging from guessing the <a href="https://www.nature.com/articles/075450a0">weight of an ox</a> or the number of <a href="https://royalsocietypublishing.org/doi/10.1098/rsif.2018.0130">jelly beans in a jar</a> to <a href="https://www.springer.com/gp/book/9783658095079">the performance of stock markets</a>.</p>
<h2>But does it work?</h2>
<p>But what happens when you start asking a crowd questions about <a href="https://journals.sagepub.com/doi/abs/10.1177/0956797614524255">events of high socioeconomic or political importance</a>? Does a crowd measure up to professional analysts – and how <a href="https://www.apa.org/pubs/journals/releases/xap-0000040.pdf">does psychology interact with their forecasting ability</a>?</p>
<p>These were some of the questions posed by the <a href="https://www.iarpa.gov/index.php/newsroom/iarpa-in-the-news/2015/439-the-good-judgment-project">Good Judgment Project</a>. Sponsored by IARPA, the US intelligence agency, it engaged thousands of people around the world to assign probabilities to the likelihood of different global events occurring, <a href="https://pubsonline.informs.org/doi/abs/10.1287/mnsc.2015.2374">using, among others, a forecasting method called prediction polls</a>. They found that the collective forecasts of the crowd were surprisingly accurate – at times outperforming those of US intelligence officers. </p>
<p>With 2019 looking so unpredictable, we at Nesta’s <a href="https://www.nesta.org.uk/blog/developing-new-centre-collective-intelligence-design-learning-how-combine-human-and-machine-intelligence-scale/">Centre for Collective Intelligence Design</a> partnered with <a href="https://www.gjopen.com/">Good Judgment Open</a> and <a href="http://www.bbc.com/future">BBC Future</a> to put crowd wisdom to the test. We wanted to see what we could learn by asking the public to forecast some major Brexit-related events.</p>
<p>To do this, we ask our crowd – anyone can sign up and so far more than 2,000 people have registered to take part worldwide – a series of questions and then judge their answers against actual events. At the halfway point of <a href="https://www.gjopen.com/challenges/34-the-2019-nesta-brexit-and-beyond-challenge-you-predict-the-future">our year-long challenge</a>, here are four of the forecasts made by our crowd so far and how accurate they’ve proved to be. </p>
<p>We asked:</p>
<p><strong>1. What will happen with Article 50 by March 30, 2019?</strong></p>
<p>What happened: The European Commission granted a conditional extension of Article 50 until October 31, 2019.</p>
<p>What the crowd said: Article 50 will be extended by the UK and the European Council (final consensus forecast: 83% probability).</p>
<figure class="align-center ">
<img alt="" src="https://images.theconversation.com/files/283986/original/file-20190714-173347-1r392xq.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=754&fit=clip" srcset="https://images.theconversation.com/files/283986/original/file-20190714-173347-1r392xq.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=600&h=323&fit=crop&dpr=1 600w, https://images.theconversation.com/files/283986/original/file-20190714-173347-1r392xq.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=600&h=323&fit=crop&dpr=2 1200w, https://images.theconversation.com/files/283986/original/file-20190714-173347-1r392xq.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=600&h=323&fit=crop&dpr=3 1800w, https://images.theconversation.com/files/283986/original/file-20190714-173347-1r392xq.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=754&h=406&fit=crop&dpr=1 754w, https://images.theconversation.com/files/283986/original/file-20190714-173347-1r392xq.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=754&h=406&fit=crop&dpr=2 1508w, https://images.theconversation.com/files/283986/original/file-20190714-173347-1r392xq.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=754&h=406&fit=crop&dpr=3 2262w" sizes="(min-width: 1466px) 754px, (max-width: 599px) 100vw, (min-width: 600px) 600px, 237px">
<figcaption>
<span class="caption"></span>
<span class="attribution"><span class="source">Green-Doe Graphic Design Ltd</span>, <span class="license">Author provided</span></span>
</figcaption>
</figure>
<p>More than 600 forecasters answered our question about the original deadline for Article 50. The question was posted in the last week of December 2018 and open for three months, but our forecasters made their collective judgement early. Already in the first week of January 2019, the crowd forecast showed that an extension of Article 50 was the most likely outcome, versus Article 50 being revoked or the UK meeting the deadline to leave the EU by March 30, 2019.</p>
<p><strong>2. What will be the closing value for the pound against the euro on April 1, 2019?</strong></p>
<p>What happened: The closing value was €1.17.</p>
<p>What the crowd said: The closing value would be between €1.10 - €1.20 (final consensus forecast: 96% probability).</p>
<figure class="align-center ">
<img alt="" src="https://images.theconversation.com/files/283987/original/file-20190714-173334-1x66eqn.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=754&fit=clip" srcset="https://images.theconversation.com/files/283987/original/file-20190714-173334-1x66eqn.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=600&h=328&fit=crop&dpr=1 600w, https://images.theconversation.com/files/283987/original/file-20190714-173334-1x66eqn.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=600&h=328&fit=crop&dpr=2 1200w, https://images.theconversation.com/files/283987/original/file-20190714-173334-1x66eqn.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=600&h=328&fit=crop&dpr=3 1800w, https://images.theconversation.com/files/283987/original/file-20190714-173334-1x66eqn.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=754&h=413&fit=crop&dpr=1 754w, https://images.theconversation.com/files/283987/original/file-20190714-173334-1x66eqn.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=754&h=413&fit=crop&dpr=2 1508w, https://images.theconversation.com/files/283987/original/file-20190714-173334-1x66eqn.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=754&h=413&fit=crop&dpr=3 2262w" sizes="(min-width: 1466px) 754px, (max-width: 599px) 100vw, (min-width: 600px) 600px, 237px">
<figcaption>
<span class="caption"></span>
<span class="attribution"><span class="source">Green-Doe Graphic Design Ltd</span>, <span class="license">Author provided</span></span>
</figcaption>
</figure>
<p>This question was live on the Good Judgment platform for 67 days from January 24, 2019. When it came to predicting the performance of the pound versus the euro, our forecasters assigned a probability on the “right side of maybe” (more than 50%) that the exchange rate would be between €1.10-€1.20 on 62 days over this three-month period. The probability for the option containing the correct final exchange rate didn’t dip below 60% after February 20, 2019, more than a full month before the original Article 50 deadline which many feared would cause trouble for the pound.</p>
<p><strong>3. In the European Parliament elections: a) What percentage of votes will the Change UK Party win? b) What percentage of votes will the Brexit Party win?</strong></p>
<p>What happened: the Brexit Party and Change UK received 30.74% and 3.31% of the vote share respectively.</p>
<p>What the crowd said: The most likely vote share for the Brexit Party would be between 30% and 35%. A vote share of less than 5% was most probable for Change UK.</p>
<p>These two questions had the quickest turnaround, they were open for the three weeks leading up to the May 22 election date. In both cases, after initial periods of high fluctuation, the crowd assigned the highest probability to the option containing the “winning” vote-share percentage almost a whole week before the public vote on May 22.</p>
<p>We saw some interesting differences when we compared our crowd’s predictions for the Brexit Party vote share to the betting exchange platform, Smarkets (which uses the prediction market approach to forecasting). The Smarkets crowd assigned a much higher likelihood to a 35%+ vote share for the <a href="https://smarkets.com/event/2147914/">Brexit Party</a> (40% at closing) whereas our crowd were much more conservative and only estimated a 17% probability for that outcome.</p>
<p>On the other hand, the Smarkets crowd was considerably more confident than our crowd when it came to <a href="https://smarkets.com/event/2147897/">Change UK</a> and closed with a 77% probability that they would win less than 5% vote share (our crowd said 55%).</p>
<p><strong>4. Will the UK have a new prime minister by July 1, 2019?</strong></p>
<p>What happened: the new prime minister for the UK is expected to be announced in the week beginning July 22.</p>
<p>What the crowd said: “No” with a closing consensus probability of 82%.</p>
<figure class="align-center ">
<img alt="" src="https://images.theconversation.com/files/283989/original/file-20190714-173338-1y752lw.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=754&fit=clip" srcset="https://images.theconversation.com/files/283989/original/file-20190714-173338-1y752lw.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=600&h=346&fit=crop&dpr=1 600w, https://images.theconversation.com/files/283989/original/file-20190714-173338-1y752lw.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=600&h=346&fit=crop&dpr=2 1200w, https://images.theconversation.com/files/283989/original/file-20190714-173338-1y752lw.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=600&h=346&fit=crop&dpr=3 1800w, https://images.theconversation.com/files/283989/original/file-20190714-173338-1y752lw.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=754&h=434&fit=crop&dpr=1 754w, https://images.theconversation.com/files/283989/original/file-20190714-173338-1y752lw.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=754&h=434&fit=crop&dpr=2 1508w, https://images.theconversation.com/files/283989/original/file-20190714-173338-1y752lw.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=754&h=434&fit=crop&dpr=3 2262w" sizes="(min-width: 1466px) 754px, (max-width: 599px) 100vw, (min-width: 600px) 600px, 237px">
<figcaption>
<span class="caption"></span>
<span class="attribution"><span class="source">Green-Doe Graphic Design Ltd</span>, <span class="license">Author provided</span></span>
</figcaption>
</figure>
<p>This question, first posted on December 21, 2018 and live for six months, was a race of two halves. Our crowd of more than 2,500 forecasters made a decisive push for “No” (70% probability) by the second week in April 2019. This followed a two-week period of uncertainty where Yes and No were forecast as almost equally probable after the original deadline for Article 50.</p>
<p>Given events over the past months and the imminent appointment of a new conservative prime minister, this could have easily been our first crowd upset, but our forecasters eventually proved accurate once again.</p>
<h2>What’s next?</h2>
<p>We’ve upped the stakes on the new version of our Brexit question for the October 31 deadline. Forecasters now have six options to choose between when predicting what will happen, including a general election and a people’s vote. Will the crowd get it right again? Currently, the crowd is predicting another Article 50 extension but no-deal Brexit and a general election aren’t far behind.</p>
<p>You can find all of the questions and try your own hand at forecasting by signing up at <a href="https://goodjudgment.io/nesta/register.html">You Predict 2019: Brexit and Beyond</a>.</p><img src="https://counter.theconversation.com/content/119906/count.gif" alt="The Conversation" width="1" height="1" />
<p class="fine-print"><em><span>Aleks Berditchevskaia is an employee of Nesta, an innovation foundation. </span></em></p><p class="fine-print"><em><span>Kathy Peach is an employee of Nesta, an innovation foundation. She is a member of the Labour party and a trustee of the charity Mines Advisory Group (MAG).</span></em></p>They’ve been right so far – and the crowd is now forecasting another Article 50 extension.Aleks Berditchevskaia, Senior Researcher, Centre for Collective Intelligence Design, NestaKathy Peach, Head of the Centre for Collective Intelligence Design, NestaLicensed as Creative Commons – attribution, no derivatives.tag:theconversation.com,2011:article/995652019-03-27T16:16:44Z2019-03-27T16:16:44ZPrinciple behind Google’s April Fools’ pigeon prank proves more than a joke<figure><img src="https://images.theconversation.com/files/264444/original/file-20190318-28499-66kcrw.jpg?ixlib=rb-1.1.0&rect=513%2C2506%2C3294%2C2255&q=45&auto=format&w=496&fit=clip" /><figcaption><span class="caption">Consider the wisdom of the flock.</span> <span class="attribution"><a class="source" href="https://unsplash.com/photos/hPOFScEaZcA">Zac Ong/Unsplash</a>, <a class="license" href="http://creativecommons.org/licenses/by/4.0/">CC BY</a></span></figcaption></figure><p><a href="https://archive.google.com/pigeonrank/">Google’s 2002 April Fools’ Day joke</a> purportedly disclosed that its popular search engine was not actually powered by artificial intelligence, but instead by biological intelligence. Google had deployed bunches of birds, dubbed pigeon clusters, to calculate the relative value of web pages because they proved to be faster and more reliable than either human editors or digital computers.</p>
<p>The joke hinged on the silliness of the premise – but the scenario does have more than a bit of the factual mixed in with the fanciful.</p>
<figure class="align-center zoomable">
<a href="https://images.theconversation.com/files/264443/original/file-20190318-28512-g4b3sl.png?ixlib=rb-1.1.0&q=45&auto=format&w=1000&fit=clip"><img alt="" src="https://images.theconversation.com/files/264443/original/file-20190318-28512-g4b3sl.png?ixlib=rb-1.1.0&q=45&auto=format&w=754&fit=clip" srcset="https://images.theconversation.com/files/264443/original/file-20190318-28512-g4b3sl.png?ixlib=rb-1.1.0&q=45&auto=format&w=600&h=354&fit=crop&dpr=1 600w, https://images.theconversation.com/files/264443/original/file-20190318-28512-g4b3sl.png?ixlib=rb-1.1.0&q=30&auto=format&w=600&h=354&fit=crop&dpr=2 1200w, https://images.theconversation.com/files/264443/original/file-20190318-28512-g4b3sl.png?ixlib=rb-1.1.0&q=15&auto=format&w=600&h=354&fit=crop&dpr=3 1800w, https://images.theconversation.com/files/264443/original/file-20190318-28512-g4b3sl.png?ixlib=rb-1.1.0&q=45&auto=format&w=754&h=445&fit=crop&dpr=1 754w, https://images.theconversation.com/files/264443/original/file-20190318-28512-g4b3sl.png?ixlib=rb-1.1.0&q=30&auto=format&w=754&h=445&fit=crop&dpr=2 1508w, https://images.theconversation.com/files/264443/original/file-20190318-28512-g4b3sl.png?ixlib=rb-1.1.0&q=15&auto=format&w=754&h=445&fit=crop&dpr=3 2262w" sizes="(min-width: 1466px) 754px, (max-width: 599px) 100vw, (min-width: 600px) 600px, 237px"></a>
<figcaption>
<span class="caption">A screenshot of Google’s explanation of how PigeonRank supposedly worked.</span>
<span class="attribution"><a class="source" href="https://archive.google.com/pigeonrank/">Google</a></span>
</figcaption>
</figure>
<p>The prank had taken a page out of 20th-century behaviorist B. F. Skinner’s <a href="https://www.verywellmind.com/operant-conditioning-a2-2794863">operant conditioning</a> playbook by allegedly teaching pigeons to peck for a food reward whenever the birds detected a relevant search result.</p>
<p>It also adapted Victorian polymath Francis Galton’s <a href="https://doi.org/10.1038/075450a0">vox populi</a> – or the voice of the people – principle by purportedly putting the web search task to something of a vote. The more the flocks of pigeons pecked at a particular website, the higher it rose on the user’s results page. This so-called PigeonRank system thus rank-ordered a user’s search results in accord with the pecking order of Google’s suitably schooled birds.</p>
<p>More than a decade later, we integrated elements of this spoof into <a href="https://doi.org/10.1371/journal.pone.0141357">our own serious research project</a> using a real mini-flock of four pigeons. Our research team included <a href="https://health.ucdavis.edu/publish/providerbio/search/11653">a pathologist</a>, <a href="https://winshipcancer.emory.edu/bios/faculty/krupinski-elizabeth.html">a radiologist</a> and <a href="https://scholar.google.com/citations?user=SIl5WVYAAAAJ&hl=en">two experimental</a> <a href="https://scholar.google.com/citations?user=CiWDe9EAAAAJ&hl=en">psychologists</a>.</p>
<figure class="align-center zoomable">
<a href="https://images.theconversation.com/files/264174/original/file-20190315-28483-11dzk01.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=1000&fit=clip"><img alt="" src="https://images.theconversation.com/files/264174/original/file-20190315-28483-11dzk01.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=754&fit=clip" srcset="https://images.theconversation.com/files/264174/original/file-20190315-28483-11dzk01.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=600&h=404&fit=crop&dpr=1 600w, https://images.theconversation.com/files/264174/original/file-20190315-28483-11dzk01.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=600&h=404&fit=crop&dpr=2 1200w, https://images.theconversation.com/files/264174/original/file-20190315-28483-11dzk01.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=600&h=404&fit=crop&dpr=3 1800w, https://images.theconversation.com/files/264174/original/file-20190315-28483-11dzk01.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=754&h=507&fit=crop&dpr=1 754w, https://images.theconversation.com/files/264174/original/file-20190315-28483-11dzk01.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=754&h=507&fit=crop&dpr=2 1508w, https://images.theconversation.com/files/264174/original/file-20190315-28483-11dzk01.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=754&h=507&fit=crop&dpr=3 2262w" sizes="(min-width: 1466px) 754px, (max-width: 599px) 100vw, (min-width: 600px) 600px, 237px"></a>
<figcaption>
<span class="caption">The test chamber provided pigeons with an image to classify for the reward of a food pellet.</span>
<span class="attribution"><a class="source" href="https://doi.org/10.1371/journal.pone.0141357">PLoS ONE 10(11): e0141357</a>, <a class="license" href="http://creativecommons.org/licenses/by/4.0/">CC BY</a></span>
</figcaption>
</figure>
<p>Exploiting the well-established <a href="http://crosstalk.cell.com/blog/pigeons-arent-just-rats-with-wings">visual and cognitive prowess of pigeons</a>, we taught our birds to peck either a blue or a yellow button on a computerized touchscreen in order to categorize pathology slides that depicted either benign or cancerous human breast tissue samples.</p>
<p>In each training session, we showed pigeons several slides of each type in random order on the touchscreen. Pigeons first had to peck the pathology slide multiple times – this step encouraged the birds to study them. Then the two report buttons popped up on each side of the tissue sample. If the tissue sample looked benign and the pigeons pecked the “benign” report button or if the presented tissue sample looked malignant and the pigeons pecked the “malignant” report button, then they received a food reward. However, if the pigeons chose the incorrect report button, then no food was given.</p>
<p>After two weeks of training, the pigeons attained accuracy levels ranging between 85 and 90 percent correct. Granted, this accomplishment falls short of their reading human text – although time will tell if that too is within <a href="https://doi.org/10.1073/pnas.1607870113">the ken of pigeons</a> – but the pigeons were quite able to make such highly accurate reports despite considerable variations in the magnification of the slide images.</p>
<p>We went on to test the pigeons with brand-new images to see if the birds could reliably transfer what they had learned; this is the key criterion for claiming that they’d learned a generalized concept of “benign/malignant tissue samples.” Accuracy to the familiar training samples averaged around 85 percent correct, and accuracy to the novel testing samples was nearly as high, averaging around 80 percent correct. This high level of transfer indicates that rote memorization alone cannot explain the pigeon’s categorization proficiency.</p>
<figure class="align-center zoomable">
<a href="https://images.theconversation.com/files/264435/original/file-20190318-28487-14ykryc.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=1000&fit=clip"><img alt="" src="https://images.theconversation.com/files/264435/original/file-20190318-28487-14ykryc.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=754&fit=clip" srcset="https://images.theconversation.com/files/264435/original/file-20190318-28487-14ykryc.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=600&h=314&fit=crop&dpr=1 600w, https://images.theconversation.com/files/264435/original/file-20190318-28487-14ykryc.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=600&h=314&fit=crop&dpr=2 1200w, https://images.theconversation.com/files/264435/original/file-20190318-28487-14ykryc.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=600&h=314&fit=crop&dpr=3 1800w, https://images.theconversation.com/files/264435/original/file-20190318-28487-14ykryc.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=754&h=395&fit=crop&dpr=1 754w, https://images.theconversation.com/files/264435/original/file-20190318-28487-14ykryc.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=754&h=395&fit=crop&dpr=2 1508w, https://images.theconversation.com/files/264435/original/file-20190318-28487-14ykryc.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=754&h=395&fit=crop&dpr=3 2262w" sizes="(min-width: 1466px) 754px, (max-width: 599px) 100vw, (min-width: 600px) 600px, 237px"></a>
<figcaption>
<span class="caption">Pigeons were able to generalize the skill of classifying tissue samples.</span>
<span class="attribution"><a class="source" href="https://doi.org/10.1371/journal.pone.0141357">PLoS ONE 10(11): e0141357</a>, <a class="license" href="http://creativecommons.org/licenses/by/4.0/">CC BY</a></span>
</figcaption>
</figure>
<p>Finally, we put Google’s PigeonRank proposal to the test. With an expanded set of breast tissue samples, we assessed the accuracy of each of four pigeons against the “wisdom of the flock,” a technique we termed “flock-sourcing.” To calculate these “flock” scores, we assigned each trial a score of 100 percent if three or four pigeons correctly responded, and we assigned a score of 50 percent if two pigeons correctly responded. Three or four pigeons never incorrectly responded.</p>
<p>The accuracy scores of the four individual pigeons were 73, 79, 81 and 85 percent correct. However, the accuracy score of the “flock” was 93 percent, thereby exceeding that of every individual bird. Pigeons <a href="http://news.mit.edu/2017/algorithm-better-wisdom-crowds-0125">thus join people</a> in evidencing better wisdom from crowds. Playing on Galton’s original term, you might call this vox columbae – or the voice-of-the-pigeons principle.</p>
<p>Although all of this may seem to be a bit of feathery fluff, over the past several years our report has resonated across several fields, going beyond pathology and radiology to include the burgeoning realm of artificial intelligence. It has been recognized in several articles including one <a href="https://www.newyorker.com/magazine/2017/04/03/ai-versus-md">quoting Geoff Hinton</a>, a key figure behind modern AI: “The role of radiologists will evolve from doing perceptual things that could probably be done by a highly trained pigeon to doing far more cognitive things.” In other words, machines may eventually be programmed to match what pigeons can do, leaving the more interesting and challenging tasks to humans.</p>
<p>What began as an elaborate April Fools’ prank has thus proved to be more than a joke. Never underestimate the brains of birds. They’re really <a href="https://www.activewild.com/bird-intelligence/">brainy beasts</a>.</p><img src="https://counter.theconversation.com/content/99565/count.gif" alt="The Conversation" width="1" height="1" />
<p class="fine-print"><em><span>The authors do not work for, consult, own shares in or receive funding from any company or organization that would benefit from this article, and have disclosed no relevant affiliations beyond their academic appointment.</span></em></p>After Google suggested PigeonRank was at the root of its search function, a group of researchers put a small flock of the birds to a different classification test in real life.Edward Wasserman, Professor of Experimental Psychology, University of IowaRichard Levenson, Professor of Pathology and Laboratory Medicine, University of California, DavisVictor Navarro, Graduate Student in Psychology, University of IowaLicensed as Creative Commons – attribution, no derivatives.tag:theconversation.com,2011:article/983162018-06-26T02:08:51Z2018-06-26T02:08:51ZThe internet is terrible at answering most tough questions. Our ‘wisdom of the crowd’ tool can help<figure><img src="https://images.theconversation.com/files/223974/original/file-20180620-126550-bujrm5.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=496&fit=clip" /><figcaption><span class="caption">In 1906, English statistician Sir Francis Galton observed the median answer of 800 participants trying to guess the weight of a cow was accurate within 1% of the correct answer.</span> <span class="attribution"><span class="source">Mavis Wong/The Conversation NY-BD-CC</span></span></figcaption></figure><p>When making tough decisions, humans have long sought advice from a higher power. </p>
<p>In ancient times, it was oracles such as <a href="https://www.ancient.eu/Pythia/">Pythia at Delphi</a>. Then the enlightenment and the industrial revolution gave rise to disciplinary experts. In the digital age, algorithms and computers have become trusted advisors.</p>
<p>But the role of experts has recently come under question. Government bureaucrats are often not perceived as trustworthy. Scientists may disagree about the best course of action. Computer algorithms have come under fire for perpetuating societal stereotypes and biases.</p>
<p>To address this, we – in partnership with universities in Australia, Finland, and Japan – developed a <a href="https://doi.org/10.1145/3173574.3173850">crowd-powered system</a> to provide “decision support” for complex problems. We’re calling it “AnswerBot.”</p>
<hr>
<p>
<em>
<strong>
Read more:
<a href="https://theconversation.com/icelands-crowd-sourced-constitution-hope-for-disillusioned-voters-everywhere-67803">Iceland's crowd-sourced constitution: hope for disillusioned voters everywhere</a>
</strong>
</em>
</p>
<hr>
<h2>Not all questions are the same</h2>
<p>Consider the following two questions:</p>
<blockquote>
<p>What car do the Ghostbusters drive?</p>
<p>Which car should I buy?</p>
</blockquote>
<p>The first question is factual. The internet is great at answering such questions. Post a question like this on Facebook, Quora, or even Google it, and you’re likely to get an answer in moments.</p>
<p>The second question requires decision support, which, in the past, has been provided by experts. </p>
<p>Similar questions include:</p>
<blockquote>
<p>Which diet should I follow?</p>
<p>How can I treat my back pain?</p>
<p>How can we deal with racism?</p>
</blockquote>
<p>The internet is terrible at answering such questions. Forum-based platforms are likely to lead to heated debate at best, or a long list of random responses that are hard to interpret. Platforms like Reddit and Quora use voting to float “good answers” to the top. However, the answers might be biased, incomplete, or fail to consider the perspective of the person asking the question.</p>
<h2>Tapping into the wisdom of the crowd</h2>
<p>In 1906, the English statistician Sir Francis Galton observed a competition in the UK town of Plymouth, where the aim was to correctly guess the weight of a cow. The median guess of 800 participants was accurate within 1% of the correct answer. Indeed, it was more accurate than any single person – even the cow experts. This finding came to be known as the “wisdom of the crowd”.</p>
<p>Our tool seeks to answer questions that require decision support by tapping into the “wisdom of the crowd”. The approach is simple, yet has already produced promising results. Here’s how it works:</p>
<h3>Step 1</h3>
<p>Work out what question needs to be answered. It has to be formulated in a manner that requires decision support. Let’s assume the question is “Which car should I buy?”</p>
<h3>Step 2</h3>
<p>AnswerBot begins posing this question to human contributors. It asks them to write down possible solutions, such as Mercedes, BMW, Mazda or Ford.</p>
<h3>Step 3</h3>
<p>The system asks the contributors to include criteria that should be considered, such as safety, price, fuel consumption, acceleration and comfort.</p>
<h3>Step 4</h3>
<p>The system generates all the possible solution-criteria combinations from the previous two steps and asks the contributors: “On a scale 0-100, how would you rate [Mercedes] in terms of [safety]”? By iterating through all combinations of cars and criteria, we construct a knowledge base that captures the collective wisdom of hundreds of people.</p>
<p>Finally, our website displays various criteria (safety, price and so on). Users can tweak associated sliders based on their personal preferences, and the system recommends a set of answers, ordered by their suitability. </p>
<p>Effectively, our system solves an optimisation problem – we use machine learning to rank the most suitable answers in our knowledge base, given someone’s preferences. </p>
<figure class="align-center ">
<img alt="" src="https://images.theconversation.com/files/224600/original/file-20180625-152156-18bd6xh.png?ixlib=rb-1.1.0&q=45&auto=format&w=754&fit=clip" srcset="https://images.theconversation.com/files/224600/original/file-20180625-152156-18bd6xh.png?ixlib=rb-1.1.0&q=45&auto=format&w=600&h=663&fit=crop&dpr=1 600w, https://images.theconversation.com/files/224600/original/file-20180625-152156-18bd6xh.png?ixlib=rb-1.1.0&q=30&auto=format&w=600&h=663&fit=crop&dpr=2 1200w, https://images.theconversation.com/files/224600/original/file-20180625-152156-18bd6xh.png?ixlib=rb-1.1.0&q=15&auto=format&w=600&h=663&fit=crop&dpr=3 1800w, https://images.theconversation.com/files/224600/original/file-20180625-152156-18bd6xh.png?ixlib=rb-1.1.0&q=45&auto=format&w=754&h=833&fit=crop&dpr=1 754w, https://images.theconversation.com/files/224600/original/file-20180625-152156-18bd6xh.png?ixlib=rb-1.1.0&q=30&auto=format&w=754&h=833&fit=crop&dpr=2 1508w, https://images.theconversation.com/files/224600/original/file-20180625-152156-18bd6xh.png?ixlib=rb-1.1.0&q=15&auto=format&w=754&h=833&fit=crop&dpr=3 2262w" sizes="(min-width: 1466px) 754px, (max-width: 599px) 100vw, (min-width: 600px) 600px, 237px">
<figcaption>
<span class="caption">AnswerBot.</span>
<span class="attribution"><span class="license">Author provided</span></span>
</figcaption>
</figure>
<p>As problems become more challenging, solutions become riddled with trade offs. A person may trade off fuel efficiency to get a car that has a better safety rating. But inability to communicate these trade offs can lead to lack of trust in the decision and decision maker, or to any recommendations that one receives.</p>
<p>Our tool makes transparent the complex trade offs required decision making by involving a large number of people in the process.</p>
<h2>Our approach can work even with small numbers</h2>
<p>AnswerBot has been deployed to answer a number of “small” and “large” problems, ranging from finding a <a href="https://doi.org/10.1145/3007900">restaurant or movie</a>, to choosing a <a href="http://people.eng.unimelb.edu.au/vkostakos/files/papers/chi2018b.pdf">weight loss diet</a>, treating <a href="http://kipuriihi.org/tutki/index_en.php">lower back pain</a>, and dealing with <a href="http://stoprasismi.org">racism</a>. We have found that it is a great way to summarise a heated debate with hundreds of participants – and it is also possible to filter out “noise” or inappropriate answers. </p>
<p>By partnering with organisations and online media we can attract hundreds of contributors to build the knowledge base – although we have <a href="https://doi.org/10.14236/ewic/HCI2016.38">shown</a> that our approach can work with just tens of contributors if there is strong agreement among them. </p>
<p>And our evaluations have found the quality of generated solutions are <a href="https://doi.org/10.1145/3007900">comparable</a> to traditional online platforms like IMDB and TripAdvisor, and its recommendations <a href="https://doi.org/10.14236/ewic/HCI2016.38">are better</a> than just randomly choosing one of solutions.</p>
<hr>
<p>
<em>
<strong>
Read more:
<a href="https://theconversation.com/heres-how-citizen-scientists-assisted-with-the-disaster-response-in-the-caribbean-85418">Here's how citizen scientists assisted with the disaster response in the Caribbean</a>
</strong>
</em>
</p>
<hr>
<p>AnswerBot can also be used to test perceptions of the crowd. We asked it to answer the question “How can I treat my back pain”? </p>
<p>After collecting our knowledge base from hundreds of people, we showed our results to medical experts and gave them the opportunity to interact with the final decision support sliders. In fact, they disagreed with some of the recommendations – for example, the crowd seemed to think that surgery is a silver bullet solution to <a href="https://theconversation.com/the-guidelines-on-low-back-pain-are-clear-drugs-and-surgery-should-be-the-last-resort-94746">back pain</a>. The experts were fascinated that it revealed how patients think about back pain, and what their misconceptions are.</p>
<h2>The wisdom of multiple experts</h2>
<p>We also tried AnswerBot on medical experts themselves. </p>
<p>We conducted a <a href="https://doi.org/10.1145/3173574.3173850">study</a> where the “crowd” was actually made up of professional back pain experts. When our results came back, we found that the experts, in fact, disagreed among themselves. </p>
<p>Our decision support tool captured, in a very transparent way, the collective understanding of the experts. Any single expert could now use the tool to compare their perception against their fellow experts.</p>
<hr>
<p>
<em>
<strong>
Read more:
<a href="https://theconversation.com/by-the-masses-the-emergence-of-crowdfunded-research-in-australia-22837">By the masses: the emergence of crowdfunded research in Australia</a>
</strong>
</em>
</p>
<hr>
<p>Our system combines human intuition with machine learning to provide decision support. By enabling the manipulation of preferences, the system provides transparency in choosing solutions, and facilitates users to experiment with various trade offs before making a decision. </p>
<p>Combining the opinions of large numbers of people – experts and non-experts alike – can provide decision support for arbitrary problems.</p><img src="https://counter.theconversation.com/content/98316/count.gif" alt="The Conversation" width="1" height="1" />
<p class="fine-print"><em><span>The work described in this article was partially funded by the Academy of Finland. The researchers behind this work are considering ways to commercialise the technology.</span></em></p>We have developed a system that combines human intuition with machine learning to provide support for people making complex decisions.Vassilis Kostakos, Professor of Human Computer Interaction, The University of MelbourneLicensed as Creative Commons – attribution, no derivatives.tag:theconversation.com,2011:article/928712018-03-09T15:44:35Z2018-03-09T15:44:35ZSeven myths of the social media age<figure><img src="https://images.theconversation.com/files/209688/original/file-20180309-30994-1dsvcgx.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=496&fit=clip" /><figcaption><span class="caption">shutterstock</span> </figcaption></figure><p>The internet was expected to renew democracy, tackle the hegemony of the monopoly news providers and draw us all into a global community. Over the past six months, that idea has been undermined by a new myth which suggests that democracy is, in fact, being overturned by the spawn of the internet: Russian bots and fake news – and that news organisations are losing their power to keep people informed.</p>
<p>But – in the UK at least – there is no evidence to suggest that made-up stories from fake news sites have had any significant impact. The BBC and the mainstream media are still our major sources of information both on and offline. Research during the <a href="http://comprop.oii.ox.ac.uk/wp-content/uploads/sites/89/2017/12/Russia-and-Brexit-v27.pdf">EU referendum campaign</a> for example found that, of all Twitter links analysed, 63.9% led to stories from professional news organisations. Junk news made up around 5% of the total and there was “little evidence of Russian content”.</p>
<h2>1. The internet has improved democracy</h2>
<p>The internet was supposed to do this by breaking up the media monopolies and allowing everyone to join the conversation. However the internet always boosts the most popular voice in <a href="https://www.amazon.co.uk/Myth-Digital-Democracy-Matthew-Hindman/dp/0691138680">every niche</a>, so the biggest news providers are still the most read, and small news publications struggle for funds. More than 200 local newspapers <a href="https://www.gov.uk/government/speeches/leveson-consultation-response">have closed</a> in the UK since 2015. Certainly there is more choice if you look for it, but the biggest concern is the number of people across the world who have simply <a href="https://academic.oup.com/esr/article/28/1/110/499101">tuned out</a> altogether and choose to watch kittens and comedy rather than news. </p>
<h2>2. We are all journalists now</h2>
<p>We can all broadcast from our smart phones, but mostly we share pictures of our children. The effect of digital disruption has been that the media landscape is becoming more concentrated and the number of paid journalists is dropping as “legacy” media organisations struggle with falling revenues. But audience members have not replaced them – those smartphone witness reports, tweeted by passers-by, would vanish into the ether if they were not found and shared by a diminishing number of paid journalists. </p>
<p>On the other hand, the internet has created an army of <a href="https://www.thenational.ae/uae/social-media-influencers-what-are-they-and-how-do-they-stay-on-the-right-side-of-the-law-1.711141">social media “influencers”</a> who, if they are canny, turn themselves into “brands” which they leverage online to recommend – or sell – everything from make up to luxury cars in return for payment in kind or cash. Meanwhile, the few genuinely new voices being created online rise and fall as they have always done, clinging to the margins and hoping to get noticed in the mad rush of information.</p>
<h2>3. The many are smarter than the few</h2>
<p>Books with titles such as <a href="http://www.bbc.com/future/story/20140708-when-crowd-wisdom-goes-wrong">The Wisdom of Crowds</a> have suggested that the internet would lead to a form of pure direct democracy because, if you ask enough people a question, the answer will always be correct. </p>
<figure class="align-left ">
<img alt="" src="https://images.theconversation.com/files/209588/original/file-20180308-30972-f9r15a.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=237&fit=clip" srcset="https://images.theconversation.com/files/209588/original/file-20180308-30972-f9r15a.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=600&h=940&fit=crop&dpr=1 600w, https://images.theconversation.com/files/209588/original/file-20180308-30972-f9r15a.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=600&h=940&fit=crop&dpr=2 1200w, https://images.theconversation.com/files/209588/original/file-20180308-30972-f9r15a.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=600&h=940&fit=crop&dpr=3 1800w, https://images.theconversation.com/files/209588/original/file-20180308-30972-f9r15a.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=754&h=1181&fit=crop&dpr=1 754w, https://images.theconversation.com/files/209588/original/file-20180308-30972-f9r15a.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=754&h=1181&fit=crop&dpr=2 1508w, https://images.theconversation.com/files/209588/original/file-20180308-30972-f9r15a.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=754&h=1181&fit=crop&dpr=3 2262w" sizes="(min-width: 1466px) 754px, (max-width: 599px) 100vw, (min-width: 600px) 600px, 237px">
<figcaption>
<span class="caption">Of course, this isn’t always the case.</span>
<span class="attribution"><span class="source">James Surowiecki via Amazon</span></span>
</figcaption>
</figure>
<p>But this naive optimism did not factor in the myriad ways in which people (or in this case their data) could be manipulated. In countries with no reliable and trusted source of mainstream news, people <a href="http://money.cnn.com/interactive/media/the-macedonia-story/">make money by inventing stories</a> tailored to press buttons of fear and prejudice. </p>
<p>In the US, where news has become highly polarised and mainstream news has lost the trust of large swaths of voters, <a href="http://275rzy1ul4252pt1hv2dqyuf.wpengine.netdna-cdn.com/wp-content/uploads/2017/07/2206.pdf">researchers</a> studying the swing state of Michigan, found stories categorised as fake news were just as likely to be shared as news from professional sources in the election period of 2016. </p>
<p>But fake news is not the preserve of junk-news factories. In late February, The Sun removed from its site an entirely specious article about savings to be made from Brexit after a mauling by economist Jonathan Portes. </p>
<p><div data-react-class="Tweet" data-react-props="{"tweetId":"968595492409135105"}"></div></p>
<p>But by that time the story had already been retweeted by leading Conservative Brexiteer, Jacob Rees-Mogg, to his 121,000 followers. Rees-Mogg has not (to date) corrected or apologised for his tweet – but then he only follows five people so he may not know about his error.</p>
<h2>4. The internet has produced a ‘global village’</h2>
<p>The “global village” was the brainchild of American media scholar <a href="http://www.media-ecology.org/publications/MEA_proceedings/v2/Meyrowitz02.pdf">Marshal McLuhan</a> who – as early as 1964 – expounded the idea that in the electronic age, everyone would have access to the same information through technology. This would seem to have been borne out by the internet.</p>
<p><div data-react-class="Tweet" data-react-props="{"tweetId":"970139279010750464"}"></div></p>
<p>But evidence suggests that the centralising tendency of monopoly global media is growing. A tiny number of companies including Facebook and Google are now the gatekeepers to information across the world – and they are nearly all American. And, in emerging economies and authoritarian states, the hopes about democratising social movements are being undone by the growing <a href="http://275rzy1ul4252pt1hv2dqyuf.wpengine.netdna-cdn.com/wp-content/uploads/2017/07/Casestudies-ExecutiveSummary-1.pdf">incursion of government</a> propaganda into the online space. </p>
<h2>5. The internet brings us together</h2>
<p>There is much to be grateful for in the ways in which the internet and social media allow us to communicate laterally. It takes only seconds to communicate to thousands via WhatApp and minutes to produce a petition and upload it to Facebook. What is less certain is its ability to unite people across the boundaries of personal affiliation and to encourage genuine debate. </p>
<figure class="align-center ">
<img alt="" src="https://images.theconversation.com/files/209707/original/file-20180309-30989-1rqtljz.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=754&fit=clip" srcset="https://images.theconversation.com/files/209707/original/file-20180309-30989-1rqtljz.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=600&h=400&fit=crop&dpr=1 600w, https://images.theconversation.com/files/209707/original/file-20180309-30989-1rqtljz.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=600&h=400&fit=crop&dpr=2 1200w, https://images.theconversation.com/files/209707/original/file-20180309-30989-1rqtljz.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=600&h=400&fit=crop&dpr=3 1800w, https://images.theconversation.com/files/209707/original/file-20180309-30989-1rqtljz.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=754&h=503&fit=crop&dpr=1 754w, https://images.theconversation.com/files/209707/original/file-20180309-30989-1rqtljz.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=754&h=503&fit=crop&dpr=2 1508w, https://images.theconversation.com/files/209707/original/file-20180309-30989-1rqtljz.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=754&h=503&fit=crop&dpr=3 2262w" sizes="(min-width: 1466px) 754px, (max-width: 599px) 100vw, (min-width: 600px) 600px, 237px">
<figcaption>
<span class="caption">Quick and easy to participate in debate via smartphone or computer.</span>
<span class="attribution"><a class="source" href="https://www.shutterstock.com/image-photo/closeup-womans-hand-reading-news-on-342236936?src=vH7rq-mei6JabUHxr3q2GA-2-5">Andrey_Popov</a></span>
</figcaption>
</figure>
<p>American researchers <a href="https://www.sciencedirect.com/science/article/pii/S0747563216300656">Michael Beam, Myiah Hutchens and Jay Hmielowski</a> tried to pick apart the different effects of reading online newspapers and sharing material on social media. They found that reading online, like reading offline, increases knowledge – but, on social media, people may share without reading. This may be partly why some scholars fear that political polarisation goes hand-in-hand with <a href="https://www.researchgate.net/publication/227737642_Polarization_and_Partisan_Selective_Exposure">rising use of social media</a>.</p>
<h2>6. Nobody trusts the mainstream media</h2>
<p>When asked whether they trust the media, the tendency in many countries is to say no – but when asked whether they trust their favourite news outlet, trust levels rise <a href="https://www.routledge.com/Misunderstanding-News-Audiences-Seven-Myths-of-the-Social-Media-Era/Elvestad-Phillips/p/book/9781138215191">dramatically</a>. However in northern Europe, one factor stands out: people trust their traditional media more than they trust online and social media news sources. More importantly public broadcasting tends to be trusted across the political spectrum drawing people together rather than <a href="http://www.digitalnewsreport.org/survey/2017/polarisation-in-the-news-media-2017/">splitting them apart</a>.</p>
<h2>7. The new ‘digital generation’</h2>
<p>Here is the biggest myth of them all – that there is a new digital generation that is mistrustful of mainstream news and busy creating a more democratic, <a href="https://www.tandfonline.com/doi/full/10.1080/13621020902731116?src=recsys">and less “dutiful”</a>, “self-actualising” future. It is reassuring to think that young people have the answers and will usher in the newer, nicer world that their elders failed to produce. But none of us is born digital. Young people are no more instinctively able to navigate online than they would be able to drive a car without lessons. Exploration alone won’t teach young people how to sort out misinformation and propaganda from facts.</p>
<p>As my Norwegian co-author Eiri Elvestad and I discuss in a <a href="http://bit.ly/2E5GQWq">new book</a>, Misunderstanding News Audiences, seven myths of the social media age, technology is changing our democracy – but we are not helpless in the face of it, nor are we liberated by it. As with previous major technical shifts, we are in the processing of adapting it to our needs and that process varies according to who we are and where we live. Democracy will be strengthened if we learn how to use the internet wisely. If we leave it to the winds of the free market we may indeed find that it overwhelms us.</p><img src="https://counter.theconversation.com/content/92871/count.gif" alt="The Conversation" width="1" height="1" />
<p class="fine-print"><em><span>Angela Phillips is affiliated with the Media Reform Coalition and the Labour Party. </span></em></p>There are a lot of misconceptions about how social media is changing society. Here are some of the most important.Angela Phillips, Professor, Goldsmiths, University of LondonLicensed as Creative Commons – attribution, no derivatives.tag:theconversation.com,2011:article/767632017-05-01T19:16:17Z2017-05-01T19:16:17ZA simple reward system could make crowds a whole lot wiser<p>There’s a problem with the wisdom of crowds.</p>
<p>Market economies and democracies rely on the idea that whole populations know more about what is best for them than a small elite group. This knowledge is potentially so powerful it can even predict the future through stock markets, betting exchanges and special investment vehicles called prediction markets. </p>
<p>These markets allow people <a href="http://www.nature.com/news/the-power-of-prediction-markets-1.20820">to trade “shares” in possible future outcomes</a>, such as the winner of upcoming elections. Anyone with new information about the future has a financial incentive to spread it by buying these shares. Prediction markets now routinely inform bookmakers odds and are <a href="http://www.cnbc.com/2016/06/13/polls-say-brexit-is-likely-but-the-betting-markets-say-it-isnt.html">quoted in news coverage of elections</a> alongside more traditional opinion polls.</p>
<p>But prediction markets are having a crisis of confidence in the abilities of the crowd. They have been systematically wrong about a series of high profile political decisions, including the <a href="http://www.newstatesman.com/politics/2015/01/who-do-betting-markets-think-will-win-election">UK general election of 2015</a>, the <a href="http://www.economist.com/blogs/graphicdetail/2016/06/polls-versus-prediction-markets">Brexit referendum</a> and the <a href="http://www.cnbc.com/2016/11/07/betting-sites-see-record-wagering-on-us-presidential-election.html">US presidential election of 2016</a>. </p>
<p>We shouldn’t expect perfect accuracy on every occasion, just as we know opinion polls <a href="https://theconversation.com/can-we-trust-the-opinion-polls-in-election-2017-76496">are often flawed</a>. But to be wrong so consistently about such prominent events points to possible flaws in the assumptions we make about crowd intelligence. For example, people don’t always act on the information they have and so it might never become part of the crowd’s decision. The dynamics of crowds and markets might also stop people from paying attention to some sources of information at all.</p>
<p>However, there might be a way forward. My colleagues and I have come up with a model that overcomes this problem by giving people a incentive to seek out new sources of information, and an extra reason to share it.</p>
<p>An important question for markets is “where do individuals get their information?” <a href="https://dx.doi.org/10.1038/scientificamerican1155-31">Research shows</a> that our opinions and activities very often match those of our peers. We also tend to look for information in the most obvious places, in line with everyone else.</p>
<p>To give an example, if you look around on any public transport in the City of London you’ll probably see people holding copies of the Financial Times. This is a problem because if everyone has the same information, the crowd is no smarter than a single individual. Studies show that having a <a href="http://dx.doi.org/10.1002/ejsp.2016">diverse collection of opinions</a>, especially <a href="http://dx.doi.org/10.1126/science.1102081">including minority views</a>, is crucial for creating a smart group.</p>
<figure class="align-center ">
<img alt="" src="https://images.theconversation.com/files/167166/original/file-20170428-12987-1hba5mi.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=754&fit=clip" srcset="https://images.theconversation.com/files/167166/original/file-20170428-12987-1hba5mi.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=600&h=389&fit=crop&dpr=1 600w, https://images.theconversation.com/files/167166/original/file-20170428-12987-1hba5mi.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=600&h=389&fit=crop&dpr=2 1200w, https://images.theconversation.com/files/167166/original/file-20170428-12987-1hba5mi.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=600&h=389&fit=crop&dpr=3 1800w, https://images.theconversation.com/files/167166/original/file-20170428-12987-1hba5mi.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=754&h=489&fit=crop&dpr=1 754w, https://images.theconversation.com/files/167166/original/file-20170428-12987-1hba5mi.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=754&h=489&fit=crop&dpr=2 1508w, https://images.theconversation.com/files/167166/original/file-20170428-12987-1hba5mi.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=754&h=489&fit=crop&dpr=3 2262w" sizes="(min-width: 1466px) 754px, (max-width: 599px) 100vw, (min-width: 600px) 600px, 237px">
<figcaption>
<span class="caption">Thinking the same.</span>
<span class="attribution"><span class="source">Shutterstock</span></span>
</figcaption>
</figure>
<p>So why do we tend to narrow the sources of our opinions? One reason is because we have an innate desire to <a href="http://dx.doi.org/10.1073/pnas.1008636108">imitate our peers</a>, to behave in ways that are safe and acceptable within our community. But it may also be because of a rational, profit-seeking motivation.</p>
<p><a href="http://www.pnas.org/cgi/doi/10.1073/pnas.1618722114">We studied</a> how theoretical profit-motivated people behave when faced with the types of rewards seen in market-like situations. To do this, we created a computer simulation of a prediction market, where people received a reward for making correct predictions. Rewards were larger when fewer people guessed the right answer, just like in a prediction market or a betting exchange.</p>
<p>The reward an individual received was a fixed amount divided by the number of other people who made a correct prediction. This was supposed to give people an incentive to look for right answers that other people wouldn’t find. But we found that people still gravitated towards a very small subset of the available information – just like London bankers with their copies of the Financial Times.</p>
<p>The more complex the situation was, the smaller the percentage of available information people actually used. The problem was that the more niche, unused information, though it might be useful to the group, was so rarely useful to the individual that possessed it that there was no incentive for them to seek it out. </p>
<h2>New reward system</h2>
<p>To counter this, we created a theoretical new prediction market system, where people would only be rewarded if they expressed accurate views but were also in the minority. For example, if someone predicted that Donald Trump would win the US election, against the consensus view, they would have received a reward once the result was known. Conversely, if most people accurately predict the Conservative Party will win the upcoming UK election then they wouldn’t receive any reward.</p>
<p>We found that this “minority reward” system, which explicitly favours those who go against popular opinion if they turn out to be correct, produced much more accurate collective decisions. This was especially the case when the situations were complex, influenced by many factors.</p>
<p>Intuitively, this makes sense. If your opinion supports the existing popular view, you can’t change whether the group will be correct or not. In our model, people have an incentive to go hunting for more esoteric sources of information about possible future outcomes. For example, rather than reading the Financial Times, they might follow obscure blogs, or read local newspapers looking for information on companies in the area.</p>
<p>They know that only by finding information that very few have access to will they have a chance to correctly go against the prevailing wisdom. This encourages the whole group to bring together a much wider set of information, leading to more accurate collective decisions.</p>
<p>Our results are so far confined to a theoretical model, but they give us an insight into why current forms of prediction markets may be prone to failure, and how we might try to improve them in future. We hope that these insights will be used to create more accurate prediction markets, as we could all benefit from better collective foresight.</p>
<p>Better predictions and collective decision making could help society decide which political ideas will or won’t work. Improving the ability of stock markets to predict which companies and ideas will do well could improve the return on investment and generate greater economic growth. Even academia is a large-scale exercise in collective wisdom. If changing the way that researchers are rewarded can improve the wisdom of this crowd, it could lead to more important scientific discoveries.</p><img src="https://counter.theconversation.com/content/76763/count.gif" alt="The Conversation" width="1" height="1" />
<p class="fine-print"><em><span>Richard Mann received funding from grants from the European Research Council. </span></em></p>A new way of encouraging people to seek out unused information could improve collective decision making.Richard Mann, University Academic Fellow in Data Analytics, University of LeedsLicensed as Creative Commons – attribution, no derivatives.tag:theconversation.com,2011:article/577052016-04-14T12:30:13Z2016-04-14T12:30:13ZHow the wisdom of crowds could solve the mystery of Shakespeare’s ‘lost plays’<figure><img src="https://images.theconversation.com/files/118549/original/image-20160413-23619-190qdft.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=496&fit=clip" /><figcaption><span class="caption">Crowdwise.</span> <span class="attribution"><a class="source" href="http://www.shutterstock.com/dl2_lim.mhtml?src=y6HXIhRzo6uD7KHiImkSXQ-1-10&clicksrc=download_btn_inline&id=304223249&size=medium_jpg&submit_jpg=">Shutterstock</a></span></figcaption></figure><p>Are the many wiser than the few? Are the masses cleverer than the expert? This is a question which has in recent years attracted an explosion of interest, perhaps most famously popularised by James Surowiecki’s 2004 book, <a href="http://www.amazon.co.uk/The-Wisdom-Crowds-Many-Smarter/dp/0349116059">The Wisdom of Crowds: Why the Many Are Smarter Than the Few</a>. </p>
<p>The idea is often traced to a <a href="http://wisdomofcrowds.blogspot.co.uk/2009/12/vox-populi-sir-francis-galton.html">paper published in 1907</a> in the science journal, Nature, by <a href="http://www.britannica.com/biography/Francis-Galton">Sir Francis Galton</a>, cousin of Charles Darwin and all-round polymath. In that paper, titled Vox Populi (Voice of the People), he demonstrated the results of a simple averaging of all the entries into a competition to guess the weight of an ox at a country fair.</p>
<p>He found that that the average (mean) guess was 1,197 pounds, just one pound off the actual weight of 1,198 pounds. This power of “crowd wisdom” to outperform even expert individuals has been demonstrated in numerous examples since, from locating a <a href="https://leightonvw.com/2011/11/22/can-prediction-markets-help-locate-a-missing-submarine/">submarine missing in the Atlantic Ocean</a> to predicting the outcome of <a href="http://onlinelibrary.wiley.com/doi/10.1002/for.2377/abstract">national elections</a> and even <a href="http://onlinelibrary.wiley.com/doi/10.1002/for.2339/abstract">papal conclaves</a>. </p>
<h2>Harnessing the power of the mob</h2>
<p>Nowadays, the way that we often harness the wisdom of the crowd is through the use of <a href="http://www.investopedia.com/terms/p/prediction-market.asp">prediction markets</a>, which are speculative markets created or employed for the purpose of aggregating information and making forecasts. These markets have been used to <a href="http://bit.ly/1p0AT0d">predict uncertain outcomes</a> ranging from the spread of infectious diseases to the demand for hospital services to the box office success of movies, to the probability of meeting internal project deadlines. The insights gained also have many potentially valuable applications for public policy more generally. </p>
<p>It’s hard to beat the fascination of watching the power of crowd wisdom, however, when it’s demonstrated in the raw, when an actual crowd demonstrates its dominance over the experts. We can go back a lot further than the days of Galton’s ox for a powerful example, and it involves a late 18th-century tale of William Shakespeare and a “lost play”. The play was called <a href="http://d.lib.rochester.edu/camelot/text/ireland-vortigern">Vortigern and Rowena</a>, and was widely proclaimed as a lost work of the Bard. </p>
<figure class="align-center ">
<img alt="" src="https://images.theconversation.com/files/118550/original/image-20160413-23631-kalrrc.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=754&fit=clip" srcset="https://images.theconversation.com/files/118550/original/image-20160413-23631-kalrrc.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=600&h=400&fit=crop&dpr=1 600w, https://images.theconversation.com/files/118550/original/image-20160413-23631-kalrrc.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=600&h=400&fit=crop&dpr=2 1200w, https://images.theconversation.com/files/118550/original/image-20160413-23631-kalrrc.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=600&h=400&fit=crop&dpr=3 1800w, https://images.theconversation.com/files/118550/original/image-20160413-23631-kalrrc.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=754&h=503&fit=crop&dpr=1 754w, https://images.theconversation.com/files/118550/original/image-20160413-23631-kalrrc.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=754&h=503&fit=crop&dpr=2 1508w, https://images.theconversation.com/files/118550/original/image-20160413-23631-kalrrc.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=754&h=503&fit=crop&dpr=3 2262w" sizes="(min-width: 1466px) 754px, (max-width: 599px) 100vw, (min-width: 600px) 600px, 237px">
<figcaption>
<span class="caption">Making sense of the past: William Shakespeare.</span>
<span class="attribution"><a class="source" href="http://www.shutterstock.com/dl2_lim.mhtml?src=nnRJ-MqRjdf1sTTNAicNlw-1-62&clicksrc=download_btn_inline&id=154066913&size=huge_jpg&submit_jpg=">Shutterstock</a></span>
</figcaption>
</figure>
<p>It was championed by <a href="http://www.jamesboswell.info/aboutjb">James Boswell</a>, the acclaimed diarist and biographer of Samuel Johnson; <a href="http://www.britannica.com/biography/Henry-James-Pye">Henry James Pye</a>, the poet laureate; another noted poet, <a href="https://en.wikipedia.org/wiki/Sir_James_Lamb,_1st_Baronet">James Bland Burges</a>; critic and classical scholar <a href="https://en.wikipedia.org/wiki/Joseph_Warton">Joseph Warton</a>, and the playwright <a href="https://en.wikipedia.org/wiki/Richard_Brinsley_Sheridan">Richard Brinsley Sheridan</a>. To widespread delight, Vortigern and Rowena opened to a packed, enthusiastic audience on the evening of April 2, 1796. The part of Vortigern himself was played by no less a light than the acclaimed Shakespearean actor, <a href="http://www.britannica.com/biography/John-Philip-Kemble">John Philip Kemble</a>. </p>
<p>The widespread excitement and anticipation among the audience soon turned, however, to bemusement and then literal disbelief, so that by the time Kemble was drawn to hint at his own opinion, repeating with emphasis Vortigern’s line “and when this solemn mockery is o’er”, the catcalls of the audience told their own story. One performance before a crowd of ordinary theatregoers was enough to kill off any notion that this was a genuine work of the Bard of Avon. The real author, <a href="http://www.npr.org/templates/story/story.php?storyId=127931669">William-Henry Ireland</a>, soon admitted to the hoax and promptly left for France. </p>
<h2>A lesson learned</h2>
<p>So what can we learn here about the wisdom of crowds? Is it perhaps the case that Shakespeare is to be played, not read, and the 18th-century experts who examined it simply took it on trust that it would appear better when played than read? Could it be that the real experts were the performers who had played much of the canon of the authentic William Shakespeare – and that their sceptical performances tipped the wink to the theatregoing crowd? Or could it be that the crowd simply was as wise as its reaction suggests? Whatever is the case, of something we can be sure. One crowd of paying spectators was enough. Vortigern and Rowena didn’t open for a second day.</p>
<p>Which brings us forward to the modern day and the strange cases of two other “lost plays”, <a href="http://britishtheatre.com/review-loves-labours-won-royal-shakespeare-theatre-4stars/">Love’s Labour’s Won</a> and <a href="http://www.theguardian.com/stage/theatreblog/2011/nov/18/cardenio-shakespeares-lost-play">Cardenio</a>. The former has never been found, but is regarded by some experts as never lost but simply an alternative name for an existing play. This was an idea picked up by the <a href="https://www.rsc.org.uk/much-ado-about-nothing/past-productions/christopher-luscombe-2014-production">Royal Shakespeare Company in 2014</a>, in their performance of Much Ado About Nothing under that name. </p>
<p>There has also been a great deal of recent scholarly interest in the idea that the early 18th-century play, <a href="http://www.dailymail.co.uk/news/article-1258116/Shakespeares-lost-play-published--just-Double-Falsehood.html">Double Falsehood</a>, was penned in large part by William Shakespeare, and is actually the lost play Cardenio, or at least closely based on it. How much truth is there in this idea? Although there is increasing support for the theory, the experts are divided. So how can we solve the mystery of such “lost plays”? Perhaps the best way is to stage them – and leave the final judgement to the wisdom of the crowd.</p><img src="https://counter.theconversation.com/content/57705/count.gif" alt="The Conversation" width="1" height="1" />
<p class="fine-print"><em><span>Leighton Vaughan Williams does not work for, consult, own shares in or receive funding from any company or organisation that would benefit from this article, and has disclosed no relevant affiliations beyond their academic appointment.</span></em></p>The mob is a surprisingly clever thing.Leighton Vaughan Williams, Professor of Economics and Finance and Director, Betting Research Unit & Political Forecasting Unit, Nottingham Trent UniversityLicensed as Creative Commons – attribution, no derivatives.tag:theconversation.com,2011:article/527742016-02-09T15:31:50Z2016-02-09T15:31:50ZHow to unleash the wisdom of crowds<figure><img src="https://images.theconversation.com/files/110195/original/image-20160203-5826-plju5v.jpg?ixlib=rb-1.1.0&rect=0%2C0%2C2048%2C1364&q=45&auto=format&w=496&fit=clip" /><figcaption><span class="caption">Bright Bunch.</span> <span class="attribution"><a class="source" href="https://www.flickr.com/photos/thomashawk/11271506886/in/photolist-ib2qiC-dptitE-epEW9-pSuBFc-snswAz-D6bkR-4z5wrn-nBAssu-4z9Kkf-sasXwh-29RVSQ-qNvk35-58q8ah-rhfPud-knkntv-hmiMnM-5kQ2qB-2Ygnw8-5ePpuR-pbe8Tg-sayXss-9ac5NE-a1GRJJ-ajkdyA-sqwwKh-eiyjgM-kqVK48-ag7khm-7VtkuP-hLFgsE-8vsCQR-e42ATy-f9eH1f-bDCd8n-pSMqRU-gNiGxC-oVf1rr-8vwHnS-nxiAfu-81JzLb-jUGF8P-9S4SKZ-qobLfN-58cc25-b6tXoZ-eTPVby-rR2sdd-5BjgFW-okDcyH-qF2ksg">Thomas Hawk</a>, <a class="license" href="http://creativecommons.org/licenses/by-nc/4.0/">CC BY-NC</a></span></figcaption></figure><p>The great Victorian polymath, <a href="http://www.galton.org/">Sir Francis Galton</a> was at a country fair in 1906, so the story goes, and came across a competition where you had to guess the weight of an ox. Once the competition was over Galton, an explorer, meteorologist, scientist and statistician, took the 787 guesses and calculated the average, which came to 1,197 pounds. The actual weight of the ox was 1,198 pounds. In effect, the crowd had provided a near perfect answer. Galton would later <a href="http://dx.doi.org/10.1038/075450a0">publish this insight in Nature</a>.</p>
<p>This phenomenon, where collective wisdom is better than most, if not all of the individuals in the crowd has become known as the Wisdom of Crowds. The authoritative take on it <a href="http://www.goodreads.com/book/show/68143.The_Wisdom_of_Crowds">came from James Surowiecki</a>. A more up-to-date example is the “Ask the Audience” part of <a href="http://www.telegraph.co.uk/culture/books/3620109/Always-ask-the-audience.html">Who Wants to be a Millionaire</a>, where the studio audience are polled and the most popular answer is the correct answer 91% of the time.</p>
<p>Even if there is a better individual guess, you face the problem of deciding which individual’s guess to select. If you choose the crowd’s guess, the decision is made for you and there is every opportunity that you will get a good answer, certainly better than choosing randomly from the other guesses. The technique has practical uses beyond the quiz show.</p>
<iframe src="https://embed-ssl.ted.com/talks/james_surowiecki_on_the_turning_point_for_social_media.html" width="100%" height="360" frameborder="0" scrolling="no" webkitallowfullscreen="" mozallowfullscreen="" allowfullscreen=""></iframe>
<h2>Understanding the Challenger Disaster</h2>
<p>On January 28 1986 the space shuttle <a href="https://theconversation.com/thirty-years-on-what-the-challenger-disaster-meant-for-our-race-into-space-53194">Challenger</a> broke up 73 seconds after launch, killing all seven astronauts on board. The disaster has been well reported in the intervening 30 years, but one intriguing aspect of it may have passed you by.</p>
<p>Almost immediately after the explosion, investors started selling stocks of the four main contractors involved in the Challenger launch – Lockheed, Rockwell International, Martin Marietta and Morton Thiokol. Of the four companies Morton Thiokol fell the most, almost 12% by the end of trading on that day, compared to about 3% for the other three companies. </p>
<p>This was a sign that the stock market felt that Morton Thiokol was to blame for the disaster but without having any firm evidence to hand. </p>
<p>In any case, six months later, the market was proven to be right. The O-ring seals on the booster rockets made by Thiokol were the cause of the problem. Richard Feynman, the renowned physicist, famously presented his findings to the Rogers Commission showing how the seals had failed.</p>
<figure>
<iframe width="440" height="260" src="https://www.youtube.com/embed/8qAi_9quzUY?wmode=transparent&start=0" frameborder="0" allowfullscreen=""></iframe>
</figure>
<p>It is still not quite clear how the wisdom of crowds managed to identify the company that was to blame for the disaster within minutes of it happening. Markets always weigh up a variety of factors and it’s hard to unpick the rationales at play. It’s just about possible that a few investors caught wind of whispers from before the launch <a href="http://spaceflightnow.com/2016/01/27/reporters-remember-challenger-coverage/">about engineers’ concerns</a>.</p>
<h2>Finding the Scorpion submarine</h2>
<p>On May 22 1968 the US navy lost one of its submarines and wanted to find the wreckage, but the intelligence it had was not able to provide an area that was small enough to effectively search. John Craven a naval officer, <a href="http://wisdomofcrowds.blogspot.co.uk/2009/12/introduction-part-v.html">decided to harness the wisdom of crowds</a>. </p>
<p>He asked a wide group of individuals, drawn from diverse backgrounds ranging from mathematicians to salvage experts to guess the submarine’s location. The group’s average guess was just 220 yards from the location where the Scorpion was eventually found.</p>
<figure>
<iframe width="440" height="260" src="https://www.youtube.com/embed/NJWHiPSvzh8?wmode=transparent&start=0" frameborder="0" allowfullscreen=""></iframe>
</figure>
<h2>What makes it work?</h2>
<p>The wisdom of crowds might seem like an easy way to to get answers. Simply ask a lot of people want they think, and aggregate the answers. If the method could find the Scorpion submarine, then a missing plane should be just as easy? Well, no.</p>
<p>As yet, nobody has been able to find the Malaysia Airlines plane MH370 that went missing in March 2014. Almost two years on and the crash site – assuming it crashed – <a href="http://gpsworld.com/post-mortem-on-flight-mh370-crowdsource-search/">has not been found</a>. That’s despite a massive crowdsourcing effort to identify the location of the aircraft, which was detailed in <a href="https://theconversation.com/crowdsourcing-hunt-for-mh370-extends-to-millions-of-sq-miles-24494">an article on The Conversation</a>. But this was a case of searching for pieces of debris, not making educated guesses about location. And it leads us in to the key rules to follow if you want to use the wisdom of crowds to your advantage.</p>
<p>Four criteria are important in making this an effective tool.</p>
<ol>
<li><p><strong>Independence</strong>: The various guesses have to be independent of one another. That is, each person must guess without the knowledge of what other people have guessed.</p></li>
<li><p><strong>Diversity</strong>: It is important to have a diverse set of guesses. In the guess the weight of the ox example, the people making the guesses ranged from farmers, butchers, livestock experts, housewives etc. That is, some people would be considered experts, while others would be considered as people with just a passing interest.</p></li>
<li><p><strong>Decentralisation</strong>: The people making the guesses should be able to draw on their private, local knowledge. </p></li>
<li><p><strong>Aggregation</strong>: There must be some way of aggregating the guesses into a single collective guess. In the guess the weight of the ox example, this was done by taking the average guess. This is a common method, but others may also be used.</p></li>
</ol>
<p>Philip Ball, in <a href="http://www.bbc.com/future/story/20140708-when-crowd-wisdom-goes-wrong">this BBC article</a>, highlighted flaws in the theory when studies ignore the rules. Remove independence and people start to gravitate towards a consensus which veers away from the accurate answer. Reduce diversity and respondents rely on shared biases, like a room full of football fans predicting results while burdened with the knowledge of which teams are the favourites. In other words, it helps to deploy a bit of wisdom when choosing your crowd.</p><img src="https://counter.theconversation.com/content/52774/count.gif" alt="The Conversation" width="1" height="1" />
<p class="fine-print"><em><span>Graham Kendall does not work for, consult, own shares in or receive funding from any company or organisation that would benefit from this article, and has disclosed no relevant affiliations beyond their academic appointment.</span></em></p>If you want the best tool for making educated guesses, then you’d better get the basics right.Graham Kendall, Professor of Operations Research and Vice-Provost, University of NottinghamLicensed as Creative Commons – attribution, no derivatives.