tag:theconversation.com,2011:/us/topics/black-mirror-49201/articlesBlack Mirror – The Conversation2021-11-23T21:07:30Ztag:theconversation.com,2011:article/1722152021-11-23T21:07:30Z2021-11-23T21:07:30ZRobots can be companions, caregivers, collaborators — and social influencers<figure><img src="https://images.theconversation.com/files/433246/original/file-20211122-25-1j2q59s.jpg?ixlib=rb-1.1.0&rect=20%2C0%2C6880%2C3456&q=45&auto=format&w=496&fit=clip" /><figcaption><span class="caption">Robot and artificial intelligence are poised to increase their influences within our every day lives.</span> <span class="attribution"><span class="source">(Shutterstock)</span></span></figcaption></figure><p>In the mid-1990s, there was research going on at Stanford University that would change the way we think about computers. The Media Equation experiments were simple: <a href="https://web.stanford.edu/group/cslipublications/cslipublications/site/1575860538.shtml">participants were asked to interact with a computer that acted socially for a few minutes after which, they were asked to give feedback about the interaction</a>. </p>
<p>Participants would provide this feedback either on the same computer (No. 1) they had just been working on or on another computer (No. 2) across the room. The study found that participants responding on computer No. 2 were far more critical of computer No. 1 than those responding on the same machine they’d worked on.</p>
<p>People responding on the first computer seemed to not want to <em>hurt</em> the computer’s <em>feelings</em> to its <em>face</em>, but had no problem talking about it behind its <em>back</em>. This phenomenon became known as the <a href="https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.175.1915&rep=rep1&type=pdf">computers as social actors (CASA)</a> paradigm because it showed that people are hardwired to respond socially to technology that presents itself as even vaguely social.</p>
<p><div data-react-class="InstagramEmbed" data-react-props="{"url":"https://www.instagram.com/p/CWHNVBSohsO","accessToken":"127105130696839|b4b75090c9688d81dfd245afe6052f20"}"></div></p>
<p>The CASA phenomenon continues to be explored, particularly as our technologies have become more social. As a researcher, lecturer and all-around lover of robotics, I observe this phenomenon in my work every time someone <a href="https://www.ideo.com/blog/why-its-important-to-say-please-and-thank-you-to-robots">thanks a robot</a>, <a href="https://shanesaunderson.com/2016/08/11/sexbots-and-terminators-exploring-gender-in-ai/">assigns it a gender</a> or tries to <a href="https://doi.org/10.1521/soco.2008.26.2.143">justify its behaviour using human, or anthropomorphic, rationales</a>. </p>
<p>What I’ve witnessed during my research is that while few are under any delusions that robots are people, we tend to defer to them just like we would another person.</p>
<h2>Social tendencies</h2>
<p>While this may sound like the beginnings of a <a href="https://www.netflix.com/ca/title/70264888"><em>Black Mirror</em> episode</a>, this tendency is precisely what allows us to enjoy social interactions with robots and place them in caregiver, collaborator or companion roles. </p>
<p>The positive aspects of treating a robot like a person is precisely why roboticists design them as such — we like interacting with people. As these technologies become more human-like, they become more capable of influencing us. However, if we continue to follow the current path of robot and AI deployment, these technologies could emerge as far more dystopian than utopian.</p>
<p>The Sophia robot, manufactured by Hanson Robotics, has been on <a href="https://www.cbsnews.com/news/60-minutes-charlie-rose-interviews-a-robot-sophia/"><em>60 Minutes</em></a>, received <a href="https://techcrunch.com/2017/10/26/saudi-arabia-robot-citizen-sophia/">honorary citizenship from Saudi Arabia</a>, holds a <a href="https://www.un.org/en/desa/un-robot-sophia-joins-meeting-artificial-intelligence-and-sustainable-development">title from the United Nations</a> and has gone on a <a href="https://time.com/5222769/will-smith-sophia-the-robot-online-dating/">date with actor Will Smith</a>. While Sophia undoubtedly highlights many technological advancements, few surpass Hanson’s achievements in marketing. If Sophia truly were a person, we would acknowledge its role as an <em>influencer</em>.</p>
<p>However, worse than robots or AI being <a href="https://www.forbes.com/sites/robertzafft/2021/02/14/will-artificial-intelligence-produce-synthetic-sociopaths/?sh=749a193d7d6e">sociopathic agents</a> — goal-oriented without morality or human judgment — these technologies become tools of mass influence for whichever organization or individual controls them.</p>
<p>If you thought the <a href="https://www.nytimes.com/2018/04/04/us/politics/cambridge-analytica-scandal-fallout.html">Cambridge Analytica scandal</a> was bad, imagine what Facebook’s algorithms of influence could do if they had an accompanying, human-like face. Or a thousand faces. Or a million. The true <a href="https://www.mirror.co.uk/tech/meet-robot-influencers-who-earning-22552147">value of a persuasive technology</a> is not in its cold, calculated efficiency, but its scale.</p>
<figure>
<iframe width="440" height="260" src="https://www.youtube.com/embed/BhU9hOo5Cuc?wmode=transparent&start=0" frameborder="0" allowfullscreen=""></iframe>
<figcaption><span class="caption">CAPTION.</span></figcaption>
</figure>
<h2>Seeing through intent</h2>
<p>Recent scandals and exposures in the tech world have left many of us feeling helpless against these corporate giants. Fortunately, <a href="https://doi.org/10.1080/09540091.2017.1313816">many of these issues can be solved through transparency</a>. </p>
<p>There are fundamental questions that are important for social technologies to answer because we would expect the same answers when interacting with another person, albeit often implicitly. Who owns or sets the mandate of this technology? What are its objectives? What approaches can it use? What data can it access? </p>
<p>Since robots could have the potential to soon <a href="https://www.nickbostrom.com/views/superintelligence.pdf">leverage superhuman capabilities</a>, enacting the will of an unseen owner, and without showing verbal or non-verbal cues that shed light on their intent, we must demand that these types of questions be answered explicitly.</p>
<p>As a roboticist, I get asked the question, “When will robots take over the world?” so often that I’ve developed a stock answer: “As soon as I tell them to.” However, my joke is underpinned by an important lesson: don’t scapegoat machines for decisions made by humans. </p>
<p>I consider myself a robot sympathizer because I think robots get unfairly blamed for many human decisions and errors. It is important that we periodically remind ourselves that a robot is not your friend, your enemy or anything in between. A robot is a tool, wielded by a person (however far removed), and increasingly used to influence us.</p><img src="https://counter.theconversation.com/content/172215/count.gif" alt="The Conversation" width="1" height="1" />
<p class="fine-print"><em><span>Shane receives funding from the Natural Sciences and Engineering Research Council of Canada (NSERC). He is affiliated with the Human Futures Institute, a Toronto-based think tank. </span></em></p>With advances in technology, robots and artificial intelligence have increasingly more sophisticated encounters with humans.Shane Saunderson, Ph.D. Candidate, Robotics, University of TorontoLicensed as Creative Commons – attribution, no derivatives.tag:theconversation.com,2011:article/1558142021-06-29T08:47:20Z2021-06-29T08:47:20ZHow a Soviet miner from the 1930s helped create today’s intense corporate workplace culture<p>One summer night in August, 1935, a young Soviet miner named Alexei Stakhanov managed to extract 102 tonnes of coal in a single shift. This was nothing short of extraordinary (according to Soviet planning, the official average for a single shift was seven tonnes). </p>
<p>Stakhanov shattered this norm by a staggering 1,400%. But the sheer quantity involved was not the whole story. It was Stakhanov’s achievement as an individual that became the most meaningful aspect of this episode. And the work ethic he embodied then – which spread all over the USSR – has been invoked by managers in the west ever since.</p>
<hr>
<iframe id="noa-web-audio-player" style="border: none" src="https://embed-player.newsoveraudio.com/v4?key=x84olp&id=https://theconversation.com/how-a-soviet-miner-from-the-1930s-helped-create-todays-intense-corporate-workplace-culture-155814&bgColor=F5F5F5&color=D8352A&playColor=D8352A" width="100%" height="110px"></iframe>
<p><em>You can listen to more articles from The Conversation, narrated by Noa, <a href="https://theconversation.com/uk/topics/audio-narrated-99682">here</a>.</em></p>
<hr>
<p>Stakhanov’s personal striving, commitment, potential and passion led to the emergence of a new ideal figure in the imagination of Stalin’s Communist Party. He even made the cover of Time magazine in 1935 as the figurehead of a new workers movement dedicated to increasing production. Stakhanov became the embodiment of a new human type and the beginning of a new social and political trend known as “Stakhanovism”. </p>
<figure class="align-right ">
<img alt="Black and white image of a man on Time magazine." src="https://images.theconversation.com/files/390621/original/file-20210319-17-7xngf7.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=237&fit=clip" srcset="https://images.theconversation.com/files/390621/original/file-20210319-17-7xngf7.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=600&h=790&fit=crop&dpr=1 600w, https://images.theconversation.com/files/390621/original/file-20210319-17-7xngf7.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=600&h=790&fit=crop&dpr=2 1200w, https://images.theconversation.com/files/390621/original/file-20210319-17-7xngf7.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=600&h=790&fit=crop&dpr=3 1800w, https://images.theconversation.com/files/390621/original/file-20210319-17-7xngf7.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=754&h=992&fit=crop&dpr=1 754w, https://images.theconversation.com/files/390621/original/file-20210319-17-7xngf7.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=754&h=992&fit=crop&dpr=2 1508w, https://images.theconversation.com/files/390621/original/file-20210319-17-7xngf7.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=754&h=992&fit=crop&dpr=3 2262w" sizes="(min-width: 1466px) 754px, (max-width: 599px) 100vw, (min-width: 600px) 600px, 237px">
<figcaption>
<span class="caption">Alexei Stakhanov on the cover of Time in 1935.</span>
<span class="attribution"><a class="source" href="http://content.time.com/time/covers/0,16641,19351216,00.html">SOVFOTO/TimeUSA</a></span>
</figcaption>
</figure>
<p>That trend still holds sway in the workplaces of today – what are human resources, after all? Management language is replete with the same rhetoric used in the 1930s by the Communist Party. It could even be argued that the atmosphere of Stakhanovite enthusiasm is even more intense today than it was in Soviet Russia. It thrives in the jargon of Human Resource Management (HRM), as its constant calls to express our passion, individual creativity, innovation and talents echo down through management structures. </p>
<p>But all this “positive” talk comes at a price. For over two decades, our research has charted the evolution of <a href="https://journals.sagepub.com/doi/10.1177/0018726708091763">managerialism</a>, <a href="https://www.tandfonline.com/doi/abs/10.1080/14797580701763855">HRM</a>, <a href="https://link.springer.com/article/10.1007/s10551-012-1436-x">employability</a> and <a href="https://rdcu.be/clN9l">performance management systems</a>, all the way through to the <a href="https://www.amazon.co.uk/Triumph-Managerialism-Technologies-Government-Implications/dp/1786604884">cultures they create</a>. We have <a href="https://www.triple-c.at/index.php/tripleC/article/view/611">shown</a> how it leaves employees with a permanent sense of never feeling good enough and the nagging worry that someone else (probably right next to us) is always <a href="https://journals.sagepub.com/doi/full/10.1177/1350508419830611">performing so much better</a>.</p>
<p>From the mid-1990s, we charted the rise of a new language for managing people – one that constantly urges us to see work as a place where we should discover “who we truly are” and express that “unique” personal “potential” which could make us endlessly “resourceful”.</p>
<p>The speed with which this language grew and spread was remarkable. But even more remarkable are the ways in which it is now spoken seamlessly in all spheres of popular culture. This is no less than the very language of the modern sense of self. And so it cannot fail to be effective. Focusing on the “self” gives management unprecedented cultural power. It intensifies work in ways which are nearly impossible to resist. Who would be able to refuse the invitation to express themselves and their presumed potential or talents?</p>
<hr>
<figure class="align-right ">
<img alt="" src="https://images.theconversation.com/files/288776/original/file-20190820-170910-8bv1s7.png?ixlib=rb-1.1.0&q=45&auto=format&w=237&fit=clip" srcset="https://images.theconversation.com/files/288776/original/file-20190820-170910-8bv1s7.png?ixlib=rb-1.1.0&q=45&auto=format&w=600&h=600&fit=crop&dpr=1 600w, https://images.theconversation.com/files/288776/original/file-20190820-170910-8bv1s7.png?ixlib=rb-1.1.0&q=30&auto=format&w=600&h=600&fit=crop&dpr=2 1200w, https://images.theconversation.com/files/288776/original/file-20190820-170910-8bv1s7.png?ixlib=rb-1.1.0&q=15&auto=format&w=600&h=600&fit=crop&dpr=3 1800w, https://images.theconversation.com/files/288776/original/file-20190820-170910-8bv1s7.png?ixlib=rb-1.1.0&q=45&auto=format&w=754&h=754&fit=crop&dpr=1 754w, https://images.theconversation.com/files/288776/original/file-20190820-170910-8bv1s7.png?ixlib=rb-1.1.0&q=30&auto=format&w=754&h=754&fit=crop&dpr=2 1508w, https://images.theconversation.com/files/288776/original/file-20190820-170910-8bv1s7.png?ixlib=rb-1.1.0&q=15&auto=format&w=754&h=754&fit=crop&dpr=3 2262w" sizes="(min-width: 1466px) 754px, (max-width: 599px) 100vw, (min-width: 600px) 600px, 237px">
<figcaption>
<span class="caption"></span>
</figcaption>
</figure>
<p><strong><em>This story is part of Conversation Insights</em></strong>
<br><em>The Insights team generates <a href="https://theconversation.com/uk/topics/insights-series-71218">long-form journalism</a> and is working with academics from different backgrounds who have been engaged in projects to tackle societal and scientific challenges.</em> </p>
<hr>
<p>Stakhanov was a kind of early poster boy for refrains like: “potential”, “talent”, “creativity”, “innovation”, “passion and commitment”, “continuous learning” and “personal growth”. They have all become the attributes management systems now hail as the qualities of ideal “human resources”. These ideas have become so entrenched in the collective psyche that many people believe they are qualities they expect of themselves, at work and <a href="https://www.amazon.co.uk/Generation-Americans-Confident-Assertive-Entitled/dp/1476755566">at home</a>. </p>
<h2>The superhero worker</h2>
<p>So, why does the spectre of this long-forgotten miner still haunt our imaginations? In the 1930s, miners lay on their sides and used picks to work the coal, which was then loaded on to carts and pulled out of the shaft by pit ponies. Stakhanov came up with some innovations, but it was his adoption of the mining drill over the pick which helped drive his productivity. The mining drill was still a novelty and required specialist training in 1930s because it was extremely heavy (more than 15kgs).</p>
<p>Once the Communist Party realised the potential of Stakhanov’s achievement, Stakhanovism took off rapidly. By the autumn of 1935, equivalents of Stakhanov emerged in every sector of industrial production. From machine building and steel works, to textile factories and milk production, record-breaking individuals were rising to the elite status of “Stakhanovite”. They were stimulated by the Communist Party’s ready adoption of Stakhanov as a leading symbol for a new economic plan. The party wanted to create an increasingly formalised elite representing the human qualities of a superhero worker. </p>
<p>Such workers began to receive special privileges (from high wages to new housing, as well as educational opportunities for themselves and their children). And so the Stakhanovites became central characters in Soviet Communist propaganda. They were showing the world what the USSR could achieve when technology was mastered by a new kind of worker who was committed, passionate, talented and creative. This new worker was promising to be the force that would propel Soviet Russia ahead of its western capitalist rivals.</p>
<p>Soviet propaganda seized the moment. A whole narrative emerged showing how the future of work and productivity in the USSR should unfold over the coming decades. Stakhanov ceased to be a person and became the human form of a system of ideas and values, outlining a new mode of thinking and feeling about work. </p>
<p>It turns out that such a story was sorely needed. The Soviet economy was not performing well. Despite gigantic investments in technological industrialisation during the so-called <a href="https://www.globalsecurity.org/military/world/russia/industry-stalin-1fyp.htm">“First Five-Year Plan”</a> (1928-1932), productivity was far from satisfactory. Soviet Russia had not overcome its own technological and economic backwardness, let alone leap over capitalist America and Europe. </p>
<h2>‘Personnel decides everything’</h2>
<p>The five-year plans were systematic programmes of resource allocation, production quotas and work rates for all sectors of the economy. The first aimed to inject the latest technology in key areas, especially industrial machine building. Its official Communist Party slogan was <em>“Technology Decides Everything”</em>. But this technological push failed to raise production; the standard of living and real wages ended up lower in 1932 than in 1928. </p>
<p>The <a href="https://www.globalsecurity.org/military/world/russia/industry-stalin-2fyp.htm">“Second Five-Year Plan”</a> (1933-1937) was going to have a new focus: <em>“Personnel Decides Everything”</em>. But not just any personnel. This was how Stakhanov stopped being a person and became an ideal type, a necessary ingredient in the recipe for this new plan.</p>
<p>On May 4, 1935, Stalin had already delivered an <a href="http://soviethistory.msu.edu/1936-2/year-of-the-stakhanovite/year-of-the-stakhanovite-texts/cadres-decide-everything/#:%7E:text=The%20slogan%20%E2%80%9CCadres%20decide%20everything%E2%80%9D%20demands%20that%20our%20leaders%20should,they%20show%20their%20first%20successes%2C">address</a> entitled <em>“Cadres [Personnel] Decide Everything”</em>. So the new plan needed figures like Stakhanov. Once he showed that it could be done, in a matter of weeks, thousands of “record-breakers” were allowed to try their hand in every sector of production. This happened despite reservations from managers and engineers who knew that machines, tools and people cannot withstand such pressures for any length of time.</p>
<p>Regardless, the party propaganda needed to let a new kind of working class elite grow as if it was spontaneous – simple workers, coming from nowhere, driven by their refusal to admit quotas dictated by the limits of machines and engineers. Indeed, they were going to show the world that it was the very denial of such limitations that constituted the essence of personal involvement in work: break all records, accept no limits, show how every person and every machine is always capable of “more”. </p>
<figure class="align-center ">
<img alt="A pamphlet with an Image of Stalin on the front." src="https://images.theconversation.com/files/405460/original/file-20210609-15107-12sfxsy.png?ixlib=rb-1.1.0&q=45&auto=format&w=754&fit=clip" srcset="https://images.theconversation.com/files/405460/original/file-20210609-15107-12sfxsy.png?ixlib=rb-1.1.0&q=45&auto=format&w=600&h=871&fit=crop&dpr=1 600w, https://images.theconversation.com/files/405460/original/file-20210609-15107-12sfxsy.png?ixlib=rb-1.1.0&q=30&auto=format&w=600&h=871&fit=crop&dpr=2 1200w, https://images.theconversation.com/files/405460/original/file-20210609-15107-12sfxsy.png?ixlib=rb-1.1.0&q=15&auto=format&w=600&h=871&fit=crop&dpr=3 1800w, https://images.theconversation.com/files/405460/original/file-20210609-15107-12sfxsy.png?ixlib=rb-1.1.0&q=45&auto=format&w=754&h=1094&fit=crop&dpr=1 754w, https://images.theconversation.com/files/405460/original/file-20210609-15107-12sfxsy.png?ixlib=rb-1.1.0&q=30&auto=format&w=754&h=1094&fit=crop&dpr=2 1508w, https://images.theconversation.com/files/405460/original/file-20210609-15107-12sfxsy.png?ixlib=rb-1.1.0&q=15&auto=format&w=754&h=1094&fit=crop&dpr=3 2262w" sizes="(min-width: 1466px) 754px, (max-width: 599px) 100vw, (min-width: 600px) 600px, 237px">
<figcaption>
<span class="caption">Stalin’s booklet on the benefits of the Stakhanov movement.</span>
<span class="attribution"><span class="source">Bogdan Costea</span>, <span class="license">Author provided</span></span>
</figcaption>
</figure>
<p>On November 17, 1935, Stalin provided a definitive explanation of Stakhanovism. Closing the First Conference of Stakhanovites of Industry and Transport of the Soviet Union, he <a href="https://www.marxists.org/reference/archive/stalin/works/1935/11/17.htm">defined</a> the essence of Stakhanovism as a leap in “consciousness” – not just a simple technical or institutional matter. Quite the contrary, the movement demanded a new kind of worker, with a new kind of soul and will, driven by the principle of unlimited progress. Stalin said:</p>
<blockquote>
<p>These are new people, people of a special type … the Stakhanov movement is a movement of working men and women which sets itself the aim of surpassing the present technical standards, surpassing the existing designed capacities, surpassing the existing production plans and estimates. Surpassing them – because these standards have already become antiquated for our day, for our new people.</p>
</blockquote>
<p>In the ensuing propaganda, Stakhanov became a symbol burdened with meanings. Ancestral hero, powerful, raw and unstoppable. But also one with a modern, rational and progressive mind which could liberate the hidden, untapped powers of technology and take command of its limitless possibilities. He was cast as a Promethean figure, leading an elite of workers whose nerves and muscles, minds and souls, were utterly attuned to the technological production systems themselves. Stakhanovism was the vision of a new humanity.</p>
<h2>‘The possibilities are endless’</h2>
<p>The Stakhanovites’ celebrity-status offered enormous ideological opportunities. It allowed the rise of production quotas. Yet this rise had to remain moderate, otherwise Stakhanovites could not be maintained as an elite. And, as an elite, Stakhanovites themselves had to be subjected to a limitation: how many top performers could really be accommodated before the very idea collapsed into normality? So quotas were engineered in a way which we might recognise today: by the <a href="https://ceopedia.org/index.php/Forced_distribution_method">forced distribution or “stack ranking”</a> of all employees according to their performance.</p>
<p>After all, how many high-performers can there be at any one time? The former CEO of General Electric, Jack Welch, <a href="https://www.shrm.org/hr-today/news/hr-magazine/pages/0603bates.aspx">suggested 20%</a> (no more, no less) every year. Indeed, the Civil Service in the UK operated on <a href="https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/971013/SCS-PM-UpdatedGuidance-April2021_1.0.pdf">this principle</a> until 2019 but used a 25% top performer quota. In 2013, Welch <a href="https://www.wsj.com/articles/SB10001424052702303789604579198281053673534">claimed</a> this system was “nuanced and humane”, that it was all “about building great teams and great companies through consistency, transparency and candor” as opposed to “corporate plots, secrecy or purges”. Welch’s argument was, however, always flawed. Any forced distribution system inextricably leads to exclusion and marginalisation of those who fall in the lower categories. Far from humane, these systems are always, inherently, threatening and ruthless.</p>
<p>And so Stakhanovism is still flowing through modern management systems and cultures, with their focus on employee performance and constant preoccupation with “high performing” individuals.</p>
<p>Something that often gets forgotten is that Stalinism itself was centred on an ideal of the <em>individual</em> soul and will: what is there that “I” am not able to do? Stakhanov fitted perfectly this ideal. Western culture has been telling itself the same ever since – <a href="https://www.microsoft.com/en-us/microsoft-365/blog/365-new-days-endless-possibilities/">“the possibilities are endless”</a>. </p>
<p>This was the logic of the Stakhanovite Movement in the 1930s. But it is also the logic of contemporary popular and corporate cultures, whose messages are now everywhere. Promises that “possibilities are endless”, that potential is “limitless”, or that you can craft any future you want, can now be found in “inspirational” posts on <a href="https://www.wired.co.uk/article/inspirational-quote-industry">social media</a>, in <a href="https://www.capgemini.com/ch-en/service/technology-operations/admnext/">management consultancy speil</a> and in just about every <a href="https://www.top100graduateemployers.com/home">graduate job advertisement</a>. One management consultancy firm even calls itself <a href="https://www.infinite-possibilities.co.uk/">Infinite Possibilities</a>. </p>
<p>Indeed, these very sentences made it on to a seemingly minor coffee coaster used by Deloitte in the early 2000s for their graduate management scheme. On one side it said: “The possibilities are endless.” While on the other side, it challenged the reader to take control of destiny itself: “It’s your future. How far will you take it?”</p>
<figure class="align-center ">
<img alt="Both sides of a coaster." src="https://images.theconversation.com/files/405162/original/file-20210608-28372-15lmfea.jpeg?ixlib=rb-1.1.0&q=45&auto=format&w=754&fit=clip" srcset="https://images.theconversation.com/files/405162/original/file-20210608-28372-15lmfea.jpeg?ixlib=rb-1.1.0&q=45&auto=format&w=600&h=292&fit=crop&dpr=1 600w, https://images.theconversation.com/files/405162/original/file-20210608-28372-15lmfea.jpeg?ixlib=rb-1.1.0&q=30&auto=format&w=600&h=292&fit=crop&dpr=2 1200w, https://images.theconversation.com/files/405162/original/file-20210608-28372-15lmfea.jpeg?ixlib=rb-1.1.0&q=15&auto=format&w=600&h=292&fit=crop&dpr=3 1800w, https://images.theconversation.com/files/405162/original/file-20210608-28372-15lmfea.jpeg?ixlib=rb-1.1.0&q=45&auto=format&w=754&h=367&fit=crop&dpr=1 754w, https://images.theconversation.com/files/405162/original/file-20210608-28372-15lmfea.jpeg?ixlib=rb-1.1.0&q=30&auto=format&w=754&h=367&fit=crop&dpr=2 1508w, https://images.theconversation.com/files/405162/original/file-20210608-28372-15lmfea.jpeg?ixlib=rb-1.1.0&q=15&auto=format&w=754&h=367&fit=crop&dpr=3 2262w" sizes="(min-width: 1466px) 754px, (max-width: 599px) 100vw, (min-width: 600px) 600px, 237px">
<figcaption>
<span class="caption"></span>
<span class="attribution"><span class="source">Bogdan Costea</span>, <span class="license">Author provided</span></span>
</figcaption>
</figure>
<p>Insignificant though these objects may appear, a discerning future archaeologist would know that they carry a most fateful kind of thinking, driving employees now as much as it drove Stakhanovites. </p>
<p>But are these serious propositions, or just ironic tropes? Since the 1980s, management vocabularies have grown almost incessantly in this respect. The rapid proliferation of fashionable management trends follows the increased preoccupation with the pursuit of “endless possibilities”, of new and unlimited horizons of self-expression and self-actualisation. </p>
<p>It is in this light that we have to show our selves as worthy members of corporate cultures. Pursuing endless possibilities becomes central to our everyday working lives. The human type created by that Soviet ideology so many decades ago, now seems to gaze at us from mission statements, values and commitments in meeting rooms, headquarters and cafeterias – but also through every website and every public expression of corporate identity.</p>
<p>Stakhanovism’s essence was a new form of individuality, of self-involvement in work. And it is this form that now finds its home as much in offices, executive suites, corporate campuses, as in schools and universities. Stakhanovism has become a movement of the individual soul. But what does an office worker actually produce and what do Stakhanovites look like today?</p>
<h2>Today’s corporate Stakhanovites</h2>
<p>In 2020, the drama series, <a href="https://www.bbc.co.uk/iplayer/episodes/m000pb89/industry">Industry</a>, created by two people with direct experience of corporate workplaces, gave us a glimpse into modern Stakhanovism. It is a sensitive and detailed examination of the destinies of five graduates joining a fictional, but <a href="https://theconversation.com/goldman-sachs-banks-benefit-from-trainees-who-think-they-must-be-superhuman-to-measure-up-157662">utterly recognisable</a>, financial institution. The show’s characters become almost instantly ruthless neo-Stakhanovites. They knew and understood that it was not what they could produce that mattered for their own success, but how they performed their successful and cool personas on the corporate stage. It was not what they did but how they appeared that mattered. </p>
<figure>
<iframe width="440" height="260" src="https://www.youtube.com/embed/PbM84z4XYgc?wmode=transparent&start=0" frameborder="0" allowfullscreen=""></iframe>
</figure>
<p>The dangers of failing to appear extraordinary, talented or creative were significant. The series showed how working life descends into unending personal, private and public struggles. In them, every character loses a sense of direction and personal integrity. Trust disappears and their very sense of self increasingly dissolves.</p>
<p>Normal days of work, normal shifts, no longer exist. Workers have to perform endlessly, gesturing so that they look committed, passionate and creative. These things are compulsory if employees are to retain some legitimacy in the workplace. So working life carries the weight of potentially determining a person’s sense of worth in every glance exchanged and in every inflection of seemingly insignificant interactions – whether in a board room, over a sandwich or a cup of coffee. </p>
<p>Friendships become impossible because human connection is no longer desirable since trusting others weakens anyone whose success is at stake. Nobody wants to fall out of the Stakhanovite society of hyper-performing top talents. Performance appraisals that may lead to dismissal are a scary prospect. And this is the case both in the series and in real life. </p>
<p>The last episode of Industry culminates in half the remaining graduates getting sacked following an operation called “<a href="https://www.shrm.org/resourcesandtools/tools-and-samples/hr-qa/pages/selectingemployeesforlayoff.aspx">Reduction In Force</a>”. This is basically a drastic final performance appraisal where each employee is forced to make a public statement arguing why they should remain – much like on the reality TV series The Apprentice. In Industry, the characters’ statements are broadcast on screens throughout the building as they describe what would make them stand out from the crowd and why they are worthier than all others. </p>
<p>Reactions to Industry emerged very quickly and viewers were enthusiastic about the show’s realism and how it resonated with their own experiences. One YouTube channel host with extensive experience of the sector reacted to each episode <a href="https://www.youtube.com/watch?v=lbKbv2X7W-U&list=PL5OrhUF4Uj2pFZhWhDSNWYAAbm-mn_mQF&index=3">in turn</a>; the <a href="https://www.ft.com/content/cd715c99-75d5-461e-ac7e-be899bc354fb">business press</a> too reacted promptly, alongside other <a href="https://www.gq-magazine.co.uk/culture/article/the-real-industry">media</a>. They converged in their conclusions: this is a serious corporate drama whose realism reveals much of the essence of work cultures today.</p>
<p>Industry is important because it touches directly on an experience so many have: the sense of a continuous competition of <a href="https://www.oxfordreference.com/view/10.1093/acref/9780199541430.001.0001/acref-9780199541430-e-372">all against all</a>. When we know that performance appraisals compare us all against each other, the consequences on mental health can be severe.</p>
<figure>
<iframe width="440" height="260" src="https://www.youtube.com/embed/R32qWdOWrTo?wmode=transparent&start=0" frameborder="0" allowfullscreen=""></iframe>
</figure>
<p>This idea is taken further in an episode of <a href="https://www.imdb.com/title/tt2085059/">Black Mirror</a>. Entitled <a href="https://www.imdb.com/title/tt5497778/">Nosedive</a>, the story depicts a world in which everything we think, feel and do becomes the object of everyone else’s rating. What if every mobile phone becomes the seat of a perpetual tribunal that decides our personal value – beyond any possibility of appeal? What if everyone around us becomes our judge? What does life feel like when all we have to measure ourselves by are other people’s instant ratings of us?</p>
<p>We asked these questions in detail in <a href="https://link.springer.com/article/10.1007/s10551-019-04399-y">our research</a> which charted the evolution of performance management systems and the cultures they create over two decades. We found that performance appraisals are becoming more public (just as in Industry), involving staff in <a href="https://www.appraisal360.co.uk/about/what-is-360-degree-feedback/">360-degree systems</a> in which every individual is rated anonymously by colleagues, managers and even clients on multiple dimensions of personal qualities. </p>
<p>Management systems focusing on <a href="https://www.shrm.org/hr-today/news/hr-magazine/pages/0615-personality-tests.aspx">individual personality</a> are now combining with the latest technologies to become <a href="https://www.questionpro.com/blog/continuous-performance-management/">permanent</a>. Ways of reporting continuously on every aspect of our personality at work are increasingly seen as central to mobilising “creativity” and “innovation”. </p>
<p>And so it might be that the atmosphere of Stakhanovite competition today is more dangerous than in 1930s Soviet Russia. It is even more pernicious because it is now driven by a confrontation between people, a confrontation between the worth of “me” against the worth of “you” as human beings – not just between the worth of what “I am able to do” against what “you are able to do”. It is a matter of a direct encounter of personal characters and their own sense of worth that has become the medium of competitive, high-performance work cultures.</p>
<p><a href="https://www.penguin.co.uk/books/196/196469/the-circle/9780241146507.html">The Circle</a>, by Dave Eggers, is perhaps the most nuanced exploration of the world of 21st-century Stakhanovism. Its characters, plot and context, its attention to detail, bring to light what it means to take up one’s personal destiny in the name of the imperative to hyper-perform and over-perform one’s self and everyone around us.</p>
<p>When the ultimate dream of becoming the central star of corporate culture comes true, a new Stakhanov is born. But who can maintain this kind of hyper-performative life? Is it even possible to be excellent, extraordinary, creative and innovative all day long? How long can a shift of performative work be anyway? The answer turns out not to be fictional at all.</p>
<h2>Stakhanovism’s limits</h2>
<p>In the summer of 2013, an intern at a major city financial institution, <a href="https://www.theguardian.com/business/2013/oct/05/moritz-erhardt-internship-banking">Moritz Erhardt</a>, was found dead one morning in the shower of his flat. It turns out that Erhardt really did try to put in a neo-Stakhanovite shift: three days and three nights of continuous work (known among London City workers and taxi drivers as a “<a href="https://www.efinancialcareers.co.uk/news/2013/08/guest-comment-interns-are-shell-shocked-by-moritz-erhardts-death-but-nothing-will-change">magic roundabout</a>”). </p>
<p><div data-react-class="Tweet" data-react-props="{"tweetId":"498450455878701056"}"></div></p>
<p>But his body could not take it. We examined this case in detail in our <a href="https://www.triple-c.at/index.php/tripleC/article/view/611">previous</a> research as well as anticipating just such a tragic scenario a year before it happened. In 2010, we reviewed a decade of the <a href="https://digital.top100graduateemployers.com/view/153459/">Times 100 Graduate Employers</a> and showed explicitly how such jobs can embody the spirit of neo-Stakhanovism. </p>
<p>Then in 2012, we <a href="https://link.springer.com/article/10.1007/s10551-012-1436-x">published our review</a> which signalled the dangers of the hyper-performative mould promoted in such publications. We argued that the graduate market is driven by an ideology of potentiality which is likely to overwhelm anyone who follows it too closely in the real world. A year later, this sense of danger became real in Erhardt’s case.</p>
<p>Stakhanov died after a stroke in Donbass, in eastern Ukraine, in 1977. A city in the region is named after him. The legacy of his achievement – or at least the propaganda that perpetuated it – lives on.</p>
<p>But the truth is that people do have limits. They do now, just as they did in the USSR in the 1930s. Possibilities are not infinite. Working towards goals of endless performance, growth and personal potential is simply not possible. Everything is finite. </p>
<p>Who we are and who we become when we work are actually fundamental and very concrete aspects of our everyday lives. Stakhanovite models of high-performance have become the register and rhythm of our working lives even though we no longer remember who Stakhanov was.</p>
<p>The danger is that we will not be able to sustain this rhythm. Just as the characters in Industry, Black Mirror or The Circle, our working lives take destructive, toxic and dark forms because we inevitably come up against the very real limits of our own purported potential, creativity or talent.</p>
<hr>
<figure class="align-center ">
<img alt="" src="https://images.theconversation.com/files/313478/original/file-20200204-41481-1n8vco4.png?ixlib=rb-1.1.0&q=45&auto=format&w=754&fit=clip" srcset="https://images.theconversation.com/files/313478/original/file-20200204-41481-1n8vco4.png?ixlib=rb-1.1.0&q=45&auto=format&w=600&h=112&fit=crop&dpr=1 600w, https://images.theconversation.com/files/313478/original/file-20200204-41481-1n8vco4.png?ixlib=rb-1.1.0&q=30&auto=format&w=600&h=112&fit=crop&dpr=2 1200w, https://images.theconversation.com/files/313478/original/file-20200204-41481-1n8vco4.png?ixlib=rb-1.1.0&q=15&auto=format&w=600&h=112&fit=crop&dpr=3 1800w, https://images.theconversation.com/files/313478/original/file-20200204-41481-1n8vco4.png?ixlib=rb-1.1.0&q=45&auto=format&w=754&h=140&fit=crop&dpr=1 754w, https://images.theconversation.com/files/313478/original/file-20200204-41481-1n8vco4.png?ixlib=rb-1.1.0&q=30&auto=format&w=754&h=140&fit=crop&dpr=2 1508w, https://images.theconversation.com/files/313478/original/file-20200204-41481-1n8vco4.png?ixlib=rb-1.1.0&q=15&auto=format&w=754&h=140&fit=crop&dpr=3 2262w" sizes="(min-width: 1466px) 754px, (max-width: 599px) 100vw, (min-width: 600px) 600px, 237px">
<figcaption>
<span class="caption"></span>
</figcaption>
</figure>
<p><em>For you: more from our <a href="https://theconversation.com/uk/topics/insights-series-71218?utm_source=TCUK&utm_medium=linkback&utm_campaign=TCUKengagement&utm_content=InsightsUK">Insights series</a>:</em></p>
<ul>
<li><p><em><a href="https://theconversation.com/cant-face-running-have-a-hot-bath-or-a-sauna-research-shows-they-offer-some-similar-benefits-158552?utm_source=TCUK&utm_medium=linkback&utm_campaign=TCUKengagement&utm_content=InsightsUK">Can’t face running? Have a hot bath or a sauna – research shows they offer some similar benefits</a></em></p></li>
<li><p><em><a href="https://theconversation.com/a-culture-of-silence-and-stigma-around-emotions-dominates-policing-officer-diaries-reveal-152657?utm_source=TCUK&utm_medium=linkback&utm_campaign=TCUKengagement&utm_content=InsightsUK">A culture of silence and stigma around emotions dominates policing, officer diaries reveal</a></em></p></li>
<li><p><em><a href="https://theconversation.com/the-double-lives-of-gay-men-in-chinas-hainan-province-153945?utm_source=TCUK&utm_medium=linkback&utm_campaign=TCUKengagement&utm_content=InsightsUK">The double lives of gay men in China’s Hainan province</a></em></p></li>
</ul>
<p><em>To hear about new Insights articles, join the hundreds of thousands of people who value The Conversation’s evidence-based news. <a href="https://theconversation.com/uk/newsletters/the-daily-newsletter-2?utm_source=TCUK&utm_medium=linkback&utm_campaign=TCUKengagement&utm_content=InsightsUK"><strong>Subscribe to our newsletter</strong></a>.</em></p><img src="https://counter.theconversation.com/content/155814/count.gif" alt="The Conversation" width="1" height="1" />
<p class="fine-print"><em><span>The authors do not work for, consult, own shares in or receive funding from any company or organisation that would benefit from this article, and have disclosed no relevant affiliations beyond their academic appointment.</span></em></p>A record-breaking Soviet miner from 1935 embodied a system of values that is central to contemporary work cultures today.Bogdan Costea, Professor of Management and Society, Lancaster UniversityPeter Watt, International Lecturer in Management and Organisation Studies, Lancaster UniversityLicensed as Creative Commons – attribution, no derivatives.tag:theconversation.com,2011:article/1600952021-05-04T02:13:10Z2021-05-04T02:13:10ZIs ‘Spot’ a good dog? Why we’re right to worry about unleashing robot quadrupeds<figure><img src="https://images.theconversation.com/files/398482/original/file-20210503-15-1grtu2b.jpg?ixlib=rb-1.1.0&rect=134%2C416%2C3664%2C2652&q=45&auto=format&w=496&fit=clip" /><figcaption><span class="caption">
</span> <span class="attribution"><span class="source">GettyImages</span></span></figcaption></figure><p>When it comes to dancing, pulling a sled, climbing stairs or doing tricks, “Spot” is definitely a good dog. It can navigate the built environment and perform a range of tasks, clearly demonstrating its flexibility as a software and hardware platform for commercial use.</p>
<p>Viral videos of Boston Dynamics’ robotic quadruped showcasing those abilities have been a key pillar of its marketing strategy. But earlier this year, when a New York art collective harnessed Spot to make a different point, the company was quick to deny its potential for harm. </p>
<p>The project, “<a href="https://spotsrampage.com/">Spot’s Rampage</a>”, involved fitting a sample of the robotic dog with a paintball gun and allowing internet users to take remote control of the creature to destroy various art works in a gallery. It ended with Spot <a href="https://nerdist.com/article/art-experiment-boston-dynamics-robot-spot-mafunction/">failing to function correctly</a>, but Boston Dynamics used Twitter to <a href="https://twitter.com/BostonDynamics/status/1362921918781943816">strongly criticise</a> the stunt:</p>
<blockquote>
<p>We condemn the portrayal of our technology in any way that promotes violence, harm, or intimidation. Our mission is to create and deliver surprisingly capable robots that inspire, delight and positively impact society.</p>
</blockquote>
<p>“Spot’s Rampage” was not the first to imagine the potential to use robot quadrupeds for violent ends. Spot also <a href="https://ew.com/tv/2017/12/29/black-mirror-metalhead-interview/">inspired</a> the “Metalhead” episode of dystopian TV series Black Mirror, in which robot quadrupeds relentlessly pursue and kill human prey. </p>
<p>This is more than science fiction, however. A serious debate over the regulation or banning of <a href="https://www.un.org/disarmament/the-convention-on-certain-conventional-weapons/background-on-laws-in-the-ccw/">lethal autonomous weapons systems</a> is happening under the auspices of the United Nations, including how such systems should comply with existing humanitarian laws. </p>
<figure>
<iframe width="440" height="260" src="https://www.youtube.com/embed/wlkCQXHEgjA?wmode=transparent&start=0" frameborder="0" allowfullscreen=""></iframe>
</figure>
<h2>Robot anxiety</h2>
<p>This contrast between the potentially violent robot of “Spot’s Rampage” and “Metalhead” and Boston Dynamics’ insistence that Spot be viewed as a force for good illustrates the tensions we have observed in our research. </p>
<p>As part of a larger project looking at <a href="https://mappinglaws.net/">debates on lethal autonomous weapons</a>, we made a detailed study of 88,970 tweets about Spot from 2007 to 2020. The results indicate public responses have been significantly less positive than Boston Dynamics would like.</p>
<hr>
<p>
<em>
<strong>
Read more:
<a href="https://theconversation.com/female-robots-are-seen-as-being-the-most-human-why-158666">Female robots are seen as being the most human. Why?</a>
</strong>
</em>
</p>
<hr>
<p>Despite the generally playful and peaceful presentation of Spot in Boston Dynamics’ videos, and obvious public interest and fascination with the technology, there is also recurring scepticism and concern from Twitter users. </p>
<p>The word cloud below maps the most commonly used negative language in those tweets. Words such as “terrifying”, “war” and “doomed” are noticeably prominent.</p>
<figure class="align-center zoomable">
<a href="https://images.theconversation.com/files/398263/original/file-20210503-13-1wupab6.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=1000&fit=clip"><img alt="" src="https://images.theconversation.com/files/398263/original/file-20210503-13-1wupab6.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=754&fit=clip" srcset="https://images.theconversation.com/files/398263/original/file-20210503-13-1wupab6.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=600&h=454&fit=crop&dpr=1 600w, https://images.theconversation.com/files/398263/original/file-20210503-13-1wupab6.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=600&h=454&fit=crop&dpr=2 1200w, https://images.theconversation.com/files/398263/original/file-20210503-13-1wupab6.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=600&h=454&fit=crop&dpr=3 1800w, https://images.theconversation.com/files/398263/original/file-20210503-13-1wupab6.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=754&h=571&fit=crop&dpr=1 754w, https://images.theconversation.com/files/398263/original/file-20210503-13-1wupab6.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=754&h=571&fit=crop&dpr=2 1508w, https://images.theconversation.com/files/398263/original/file-20210503-13-1wupab6.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=754&h=571&fit=crop&dpr=3 2262w" sizes="(min-width: 1466px) 754px, (max-width: 599px) 100vw, (min-width: 600px) 600px, 237px"></a>
<figcaption>
<span class="caption"></span>
<span class="attribution"><span class="license">Author provided</span></span>
</figcaption>
</figure>
<p>Analysis of the use of emotive words shows recurring features of conversations about Spot: dark humour, sarcasm and suspicion about the intended uses of the technology. Associations between Boston Dynamics, its previous owner Google, the military and killer robots portrayed in popular culture (such the Terminator films) also recur.</p>
<p>Boston Dynamics CEO Robert Playter has <a href="https://www.cbsnews.com/news/boston-dynamics-robots-humans-animals-60-minutes-2021-03-28/">dismissed</a> such negative public reactions as “fiction” grounded in the “rogue robot story” and misunderstandings of the technology. </p>
<p>Depictions of robots that will not harm humans and even save lives have been a mainstay of public messaging by both Boston Dynamics and the US military’s Defense Advanced Research Projects Agency. From <a href="https://www.npr.org/sections/coronavirus-live-updates/2020/04/24/844770815/meet-spot-the-robot-that-could-help-doctors-remotely-treat-covid-19-patients">fighting COVID-19</a> to <a href="https://www.subtchallenge.com/">search and rescue</a> to <a href="https://www.businessinsider.com.au/us-marines-testing-boston-dynamics-robot-called-spot-2015-9?r=US&IR=T">taking soldiers out of harm’s way</a>, the potential humanitarian applications of robotic quadrupeds in civilian and military service are emphasised.</p>
<p><div data-react-class="Tweet" data-react-props="{"tweetId":"1387471778528866310"}"></div></p>
<h2>Military connections</h2>
<p>But negative reactions should not be too easily discounted. Boston Dynamics’ technology was advanced through military funding, and military applications have been seen as a key market. In 2019, company founder and then CEO <a href="https://www.boston.com/news/technology/2019/10/28/boston-dynamics-robots-terrifying">Marc Raibert signalled</a> Boston Dynamics “will probably have military customers”. </p>
<p>And in the month following the “Spot’s Rampage” prank, the robot was <a href="https://www.theverge.com/2021/2/24/22299140/nypd-boston-dynamics-spot-robot-dog">tested by the NYPD</a> and by <a href="https://www.theverge.com/2021/4/7/22371590/boston-dynamics-spot-robot-military-exercises-french-army">French armed forces</a> in combat exercises — although public backlash against the NYPD trials have <a href="https://www.theverge.com/2021/4/29/22409559/nypd-robot-dog-digidog-boston-dynamics-contract-terminated">brought an early end</a> to its contract with Boston Dynamics.</p>
<hr>
<p>
<em>
<strong>
Read more:
<a href="https://theconversation.com/abusing-a-robot-wont-hurt-it-but-it-could-make-you-a-crueller-person-126187">Abusing a robot won't hurt it, but it could make you a crueller person</a>
</strong>
</em>
</p>
<hr>
<p>Meanwhile, other robotics companies, including Boston Dynamics’ competitor Ghost Robotics, have actively and successfully sought <a href="https://www.thedrive.com/the-war-zone/36229/the-air-force-just-tested-robot-dogs-for-use-in-base-security">contracts with the US military</a>. </p>
<p><div data-react-class="Tweet" data-react-props="{"tweetId":"1374812744709459979"}"></div></p>
<p>Ghost Robotics CEO Jiren Parikh has <a href="https://breakingdefense.com/2020/01/air-force-seeks-innovators-at-first-abms-industry-day/">said</a> its Vision 60 quadruped “can be used for anything from perimeter security to detection of chemical and biological weapons to actually destroying a target”. </p>
<p>More recently, Ghost Robotics released <a href="https://www.youtube.com/watch?v=0iOmcubN51A">footage</a> of its robot quadruped firing a projectile into a target to demonstrate its potential use for bomb disposal.</p>
<p>The apparent flexibility of these machines, which can carry different payloads and be fitted and programmed for different missions, suggests a range of potential applications, including as lethal autonomous weapons. </p>
<p>As long as the military end-use remains uncertain and the technology itself is still developing, we should remain wary of attempts by developers, marketers and military advocates to shape and manage public sentiment with the promise of “saving lives”.</p><img src="https://counter.theconversation.com/content/160095/count.gif" alt="The Conversation" width="1" height="1" />
<p class="fine-print"><em><span>Jeremy Moses receives funding from the Royal Society of New Zealand Marsden Fund. </span></em></p><p class="fine-print"><em><span>Geoffrey Ford receives funding from the Royal Society of New Zealand Marsden Fund. </span></em></p>Marketing for robotic ‘dogs’ plays up their potential for good, but the debate about lethal autonomous weapons suggests public anxiety is warranted.Jeremy Moses, Associate Professor in International Relations, University of CanterburyGeoffrey Ford, Lecturer in Digital Humanities / Postdoctoral Fellow in Political Science and International Relations, University of CanterburyLicensed as Creative Commons – attribution, no derivatives.tag:theconversation.com,2011:article/1554362021-03-01T15:01:30Z2021-03-01T15:01:30ZChatbots that resurrect the dead: legal experts weigh in on ‘disturbing’ technology<figure><img src="https://images.theconversation.com/files/385593/original/file-20210222-21-1p5c1z0.jpg?ixlib=rb-1.1.0&rect=34%2C0%2C7634%2C4320&q=45&auto=format&w=496&fit=clip" /><figcaption><span class="caption">
</span> <span class="attribution"><a class="source" href="https://www.shutterstock.com/image-illustration/virtual-mans-head-made-digital-data-1919428025">Tatiana Shepeleva/Shutterstock</a></span></figcaption></figure><p>It was recently revealed that in 2017 Microsoft <a href="https://www.independent.co.uk/life-style/gadgets-and-tech/microsoft-chatbot-patent-dead-b1789979.html">patented a chatbot</a> which, if built, would digitally resurrect the dead. Using AI and machine learning, the proposed chatbot would bring our digital persona back to life for our family and friends to talk to. When pressed on the technology, Microsoft representatives admitted that the chatbot was “<a href="https://twitter.com/_TimOBrien/status/1352645952310439936">disturbing</a>”, and that there were currently no plans to put it into production.</p>
<p>Still, it appears that the technical tools and personal data are in place to make digital reincarnations possible. AI chatbots have already passed the “<a href="https://www.bbc.co.uk/news/technology-27762088">Turing Test</a>”, which means they’ve fooled other humans into thinking they’re human, too. Meanwhile, most people in the modern world now leave behind enough data to teach AI programmes about our conversational idiosyncrasies. Convincing digital doubles may be just around the corner.</p>
<p>But there are currently no laws governing digital reincarnation. Your right to data privacy after your death is far from set in stone, and there is currently no way for you to opt out of being digitally resurrected. This legal ambiguity leaves room for private companies to make chatbots out of your data after you’re dead.</p>
<p>Our <a href="https://edinburghuniversitypress.com/book-future-law.html">research</a> has looked at the surprisingly complex legal question of what happens to your data after you die. At present, and in the absence of specific legislation, it’s unclear who might have the ultimate power to reboot your digital persona after your physical body has been put to rest. </p>
<figure class="align-center ">
<img alt="A woman lying in bed looking at a lit-up phone screen on a pillow" src="https://images.theconversation.com/files/385596/original/file-20210222-21-1p46g52.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=754&fit=clip" srcset="https://images.theconversation.com/files/385596/original/file-20210222-21-1p46g52.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=600&h=338&fit=crop&dpr=1 600w, https://images.theconversation.com/files/385596/original/file-20210222-21-1p46g52.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=600&h=338&fit=crop&dpr=2 1200w, https://images.theconversation.com/files/385596/original/file-20210222-21-1p46g52.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=600&h=338&fit=crop&dpr=3 1800w, https://images.theconversation.com/files/385596/original/file-20210222-21-1p46g52.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=754&h=424&fit=crop&dpr=1 754w, https://images.theconversation.com/files/385596/original/file-20210222-21-1p46g52.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=754&h=424&fit=crop&dpr=2 1508w, https://images.theconversation.com/files/385596/original/file-20210222-21-1p46g52.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=754&h=424&fit=crop&dpr=3 2262w" sizes="(min-width: 1466px) 754px, (max-width: 599px) 100vw, (min-width: 600px) 600px, 237px">
<figcaption>
<span class="caption">Be Right Back, an episode of the Black Mirror TV series, featured a woman addicted to a chatbot representation of her dead partner.</span>
</figcaption>
</figure>
<p><a href="https://pdfpiw.uspto.gov/.piw?PageNum=0&docid=10853717&IDKey=6E72242A6301&HomeUrl=http%3A%2F%2Fpatft.uspto.gov%2Fnetacgi%2Fnph-Parser%3FSect1%3DPTO2%2526Sect2%3DHITOFF%2526p%3D1%2526u%3D%25252Fnetahtml%25252FPTO%25252Fsearch-bool.html%2526r%3D31%2526f%3DG%2526l%3D50%2526co1%3DAND%2526d%3DPTXT%2526s1%3Dmicrosoft.ASNM.%2526OS%3DAN%2Fmicrosoft%2526RS%3DAN%2Fmicrosofthttps://pdfpiw.uspto.gov/.piw?PageNum=0&docid=10853717&IDKey=6E72242A6301&HomeUrl=http%3A%2F%2Fpatft.uspto.gov%2Fnetacgi%2Fnph-Parser%3FSect1%3DPTO2%2526Sect2%3DHITOFF%2526p%3D1%2526u%3D%25252Fnetahtml%25252FPTO%25252Fsearch-bool.html%2526r%3D31%2526f%3DG%2526l%3D50%2526co1%3DAND%2526d%3DPTXT%2526s1%3Dmicrosoft.ASNM.%2526OS%3DAN%2Fmicrosoft%2526RS%3DAN%2Fmicrosoft">Microsoft’s chatbot</a> would use your electronic messages to create a <a href="https://www.ipwatchdog.com/2021/02/09/microsoft-patent-reaches-beyond-death-pseudo-reincarnation/id=129801/">digital reincarnation</a> in your likeness after you pass away. Such a chatbot would use machine learning to respond to text messages just as you would have when you were alive. If you happen to leave behind rich voice data, that too could be used to create your vocal likeness – someone your relatives could speak with, through a phone or a humanoid robot.</p>
<p>Microsoft isn’t the only company to have shown an interest in digital resurrection. The AI company <a href="https://www.businessinsider.com/eternime-and-replika-giving-life-to-the-dead-with-new-technology-2018-11?r=US&IR=T">Eternime</a> has built an AI-enabled chatbot which harvests information – including geolocation, motion, activity, photos, and Facebook data – which lets users create an avatar of themselves to live on after they die. It may be only a matter of time until families have the choice to reanimate dead relatives using AI technologies such as Eternime’s.</p>
<hr>
<p>
<em>
<strong>
Read more:
<a href="https://theconversation.com/bereaved-who-take-comfort-in-digital-messages-from-dead-loved-ones-live-in-fear-of-losing-them-109754">Bereaved who take comfort in digital messages from dead loved ones live in fear of losing them</a>
</strong>
</em>
</p>
<hr>
<p>If chatbots and holograms from beyond the grave are set to become commonplace, we’ll need to draw up new laws to govern them. After all, it looks like a violation of the right to privacy to digitally resurrect someone whose body lies beneath a tombstone reading “rest in peace”.</p>
<h2>Bodies in binary</h2>
<p>National laws are inconsistent on how your data is used after your death. In the EU, the <a href="https://gdpr-info.eu/">law</a> on data privacy only protects the rights of the living. That leaves <a href="https://gdpr-info.eu/recitals/no-27/">room for member states</a> to decide how to protect the data of the dead. Some, such as <a href="https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3599852">Estonia, France, Italy and Latvia</a>, have legislated on postmortem data. The UK’s <a href="https://www.legislation.gov.uk/ukpga/2018/12/contents/enacted">data protection laws</a> have not.</p>
<p>To further complicate matters, our data is mostly controlled by private online platforms such as Facebook and Google. This control is based on the terms of service that we sign up to when we create profiles on these platforms. Those terms fiercely protect the privacy of the dead.</p>
<p>For example, in 2005, <a href="https://www.cnet.com/news/yahoo-releases-e-mail-of-deceased-marine/">Yahoo! refused</a> to provide email account login details for the surviving family of a US marine killed in Iraq. The company argued that their terms of service were designed to protect the marine’s privacy. A judge eventually <a href="http://usatoday30.usatoday.com/tech/news/2005-04-21-marine-e-mail_x.htm?POE=TECISVA">ordered</a> the company to provide the family with a CD containing copies of the emails, setting a legal precedent in the process. </p>
<hr>
<p>
<em>
<strong>
Read more:
<a href="https://theconversation.com/people-are-going-to-court-over-dead-family-members-facebook-pages-its-time-for-post-mortem-privacy-78375">People are going to court over dead family members' Facebook pages – it's time for post-mortem privacy</a>
</strong>
</em>
</p>
<hr>
<p>A few initiatives, such as <a href="https://www.theatlantic.com/technology/archive/2013/04/google-death-a-tool-to-take-care-of-your-gmail-when-youre-gone/274934/">Google’s Inactive Account Manager</a> and <a href="https://publications.aston.ac.uk/id/eprint/37748/1/Post_mortem_privacy_2_0_theory_law_and_technology.pdf">Facebook’s Legacy Contact</a>, have attempted to address the postmortem data issue. They allow living users to make some <a href="https://www.tandfonline.com/doi/full/10.1080/13600869.2017.1275116">decisions</a> on what happens to their data assets after they die, helping to avoid ugly court battles over dead people’s data in the future. But these measures are no substitute for laws.</p>
<p>One route to better postmortem data legislation is to follow the example of <a href="https://research.aston.ac.uk/en/publications/posthumous-medical-data-donation-the-case-for-a-legal-framework">organ donation</a>. The UK’s “opt out” <a href="https://www.organdonation.nhs.uk/uk-laws/organ-donation-law-in-england/">organ donation law</a> is particularly relevant, as it treats the organs of the dead as donated unless that person specified otherwise when they were alive. The same opt out scheme could be applied to postmortem data.</p>
<p>This model could help us respect the privacy of the dead and the wishes of their heirs, all while considering the <a href="https://link.springer.com/chapter/10.1007/978-3-030-04363-6_11#:%7E:text=Posthumous%2520medical%2520data%2520donation%2520(PMDD)%2520refers%2520to%2520the%2520act%2520of,health%2520services%252C%2520throughout%2520their%2520life.">benefits</a> that could arise from <a href="https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3536264">donated data</a>: that data donors <a href="https://ec.europa.eu/digital-single-market/en/news/proposal-regulation-european-data-governance-data-governance-act">could help save lives</a> just as organ donors do.</p>
<p>In the future, private companies may offer family members an agonising choice: abandon your loved one to death, or instead pay to have them digitally revived. Microsoft’s chatbot may at present be too disturbing to countenance, but it’s an example of what’s to come. It’s time <a href="https://thefutureofprivacylaw.wordpress.com/">we wrote the laws to govern this technology.</a></p><img src="https://counter.theconversation.com/content/155436/count.gif" alt="The Conversation" width="1" height="1" />
<p class="fine-print"><em><span>Edina Harbinja receives funding from the Leverhulme Trust for the project 'Modern Technologies, Privacy Law and the Dead', along with Lilian Edwards and Marisa McVey.</span></em></p><p class="fine-print"><em><span>Lilian Edwards receives funding from the Leverhulme Trust for the project 'Modern Technologies, Privacy Law and the Dead', along with Edina Harbinja and Marisa McVey.</span></em></p><p class="fine-print"><em><span>Marisa McVey receives funding from the Leverhulme Trust for the project 'Modern Technologies, Privacy Law and the Dead', along with Edina Harbinja and Lilian Edwards.</span></em></p>Our newfound ability to reincarnate the dead as chatbots presents several legal and ethical dilemmas.Edina Harbinja, Senior Lecturer in Media/Privacy Law, Aston UniversityLilian Edwards, Professor of Law, Innovation & Society, Newcastle Law School, Newcastle UniversityMarisa McVey, Research fellow, Aston UniversityLicensed as Creative Commons – attribution, no derivatives.tag:theconversation.com,2011:article/1305562020-02-10T10:39:53Z2020-02-10T10:39:53ZWhy we’re challenging domestic violence perpetrators with interactive storytelling<figure><img src="https://images.theconversation.com/files/313956/original/file-20200206-43113-w1ihog.jpg?ixlib=rb-1.1.0&rect=186%2C7%2C4873%2C3437&q=45&auto=format&w=496&fit=clip" /><figcaption><span class="caption">one in four perpetrators are repeat offenders, yet less than 1% of perpetrators receive a specialist intervention to challenge or change their behaviour</span> <span class="attribution"><a class="source" href="https://www.shutterstock.com/image-photo/father-son-silhouette-on-sunset-1378418711">KonstantinChristian/Shutterstock</a></span></figcaption></figure><p>Domestic violence is a common, highly damaging crime that affects <a href="https://www.who.int/news-room/fact-sheets/detail/violence-against-women">one in three women in their lifetime</a> and one in six men. The UK’s Office for National Statistics reported that over <a href="https://www.ons.gov.uk/peoplepopulationandcommunity/crimeandjustice/articles/domesticabuseprevalenceandtrendsenglandandwales/yearendingmarch2019">2.4 million adults</a> experienced domestic violence within England and Wales in 2019, that’s 5.7% of the adult population. </p>
<p>At the moment we do not have accurate statistics on how many people use domestic violence in their intimate relationships. However, <a href="http://driveproject.org.uk/wp-content/uploads/2020/01/Executive-Summary_Final2020.pdf">recent research</a> found that to reduce the number of victims we must challenge perpetrators to stop. Working with children’s charity Barnardo’s, a team of us are developing a prevention intervention that uses interactive storytelling to do just that. </p>
<p>Challenging perpetrators of domestic violence may seem like common sense. But only <a href="http://www.bristol.ac.uk/sps/news/2020/drive-call-to-action.html">1% of identified perpetrators</a> receive a specialist intervention for their behaviour. This means that many people who use violence are not effectively challenged or fail to be disciplined for their actions. As such, they continue to abuse their current (or potential future) victim-survivors. </p>
<p>Domestic violence prevention programmes are an effective way of challenging the behaviour when it occurs, ensuring the safety of victim-survivors. These programmes work with perpetrators to develop respectful, non-abusive relationships. They do so by equipping them with the understanding and tools to change their behaviour. </p>
<h2>Choice-Point</h2>
<p><a href="https://www.nature.com/articles/s41598-018-19987-7">Research has shown</a> that perpetrators often have difficulty with the idea of domestic violence as a choice. Some of them may also struggle to put themselves in the shoes of others. </p>
<p>As part of <a href="https://doi.org/10.1145/3313831.3376386">my research</a> into how to better <a href="https://digitalcivics.io/">design digital systems for citizens</a>, I ran a domestic violence prevention intervention with Barnardo’s to create an interactive digital story system. </p>
<p>Known as Choice-Point, this interactive fiction allows perpetrators to take on different roles in a family so they can reflect on their behaviour in a controlled environment.</p>
<p>Interactive fiction is re-emerging as a way of engaging users to learn about and explore serious scenarios. The most well-known example is Black Mirror’s <a href="https://variety.com/2018/digital/news/netflix-black-mirror-bandersnatch-interactive-1203096171/">Bandersnatch</a>. By providing viewers with options that affected the story’s outcomes, this TV show challenged <a href="https://dl.acm.org/doi/fullHtml/10.1145/3317697.3325124">perceptions of choice</a>, morality and use of violence.</p>
<p>Following a lengthy study with my charity partner, we performed interviews, focus groups and design workshops to design Choice-Point to be used in group interventions with perpetrators. </p>
<p>The story, co-written with the charity and victim-survivors, begins after Terry, the father, verbally degrades his female partner Sharon in front of their two children. We worked with groups of eight to ten perpetrators, each of whom played as a different character from the family or could vote on the path that they would want the story to go. </p>
<p>They all had a set of choices that had a direct impact on what the following player could have done in the story. Ultimately, these choices influenced the ending of the story, ranging from the most rewarding, where the father seeks help for his abusive behaviour, to the most sombre, resulting in further upset by the father’s violence. </p>
<figure class="align-center ">
<img alt="" src="https://images.theconversation.com/files/312543/original/file-20200129-92949-l5jhpi.png?ixlib=rb-1.1.0&q=45&auto=format&w=754&fit=clip" srcset="https://images.theconversation.com/files/312543/original/file-20200129-92949-l5jhpi.png?ixlib=rb-1.1.0&q=45&auto=format&w=600&h=468&fit=crop&dpr=1 600w, https://images.theconversation.com/files/312543/original/file-20200129-92949-l5jhpi.png?ixlib=rb-1.1.0&q=30&auto=format&w=600&h=468&fit=crop&dpr=2 1200w, https://images.theconversation.com/files/312543/original/file-20200129-92949-l5jhpi.png?ixlib=rb-1.1.0&q=15&auto=format&w=600&h=468&fit=crop&dpr=3 1800w, https://images.theconversation.com/files/312543/original/file-20200129-92949-l5jhpi.png?ixlib=rb-1.1.0&q=45&auto=format&w=754&h=588&fit=crop&dpr=1 754w, https://images.theconversation.com/files/312543/original/file-20200129-92949-l5jhpi.png?ixlib=rb-1.1.0&q=30&auto=format&w=754&h=588&fit=crop&dpr=2 1508w, https://images.theconversation.com/files/312543/original/file-20200129-92949-l5jhpi.png?ixlib=rb-1.1.0&q=15&auto=format&w=754&h=588&fit=crop&dpr=3 2262w" sizes="(min-width: 1466px) 754px, (max-width: 599px) 100vw, (min-width: 600px) 600px, 237px">
<figcaption>
<span class="caption">Choices Crop.</span>
<span class="attribution"><span class="license">Author provided</span></span>
</figcaption>
</figure>
<p>Positive choices by the story’s perpetrator permitted others in the group playing as the children or the female partner to have a wider variety of options. But negative choices by the perpetrator directly restricted the choices of the others, reflecting the impact of coercive and controlling behaviour on other people. </p>
<p>If a participant was resistant to getting involved, we also designed the option for them to participate as an anonymous audience member who could vote on their preferred mode of action through a corresponding mobile device.</p>
<h2>Reflections on Reality</h2>
<p>We trialled the system with three groups of perpetrators (a total of 27 men) that were taking part in an awareness-raising intervention on domestic violence. Our three groups played through Choice-Point twice in each program (taking around 35 minutes each), after which the research team asked the men to evaluate the tool. </p>
<p>We discovered that using a fictional narrative and constraining the sets of choices that users could select resulted in the men exhibiting positive change in how they reflected on their behaviour. This included understanding the connection between their actions, controlling the story and the use of violence in real life. </p>
<p>In some cases, the connection was so strong that the perpetrators used the characters to disclose personal and sensitive reflections on their own behaviour, blurring the barrier between fiction and reality. </p>
<p>In all run-throughs of the story, both the research team and the people running the sessions said Choice-Point provided a protective, nonjudgmental cover for the men to discuss their experiences of using violence. The session organisers said this was especially positive as they found, <a href="https://dl.acm.org/doi/10.1145/3328320.3328405">in line with other studies</a>, that ensuring non-judgemental spaces was crucial for people to learn constructively and reflect meaningfully on their behaviour.</p>
<p>Many people agree that more needs to be done to work with perpetrators of domestic violence. On 21 January 2020, several organisations including <a href="https://www.barnardos.org.uk/who-we-are">Barnardo’s</a> and the domestic abuse charity <a href="http://respect.uk.net/what-we-do/">RESPECT</a>, went to Parliament to present a <a href="http://driveproject.org.uk/wp-content/uploads/2020/01/Call-to-Action-Final.pdf">National Perpetrator Strategy</a>. Part of this includes using <a href="http://respect.uk.net/wp-content/uploads/2017/02/Respect-Standard-15.11.17.pdf">quality-assured</a> domestic violence prevention programmes.</p>
<p>Evaluations of the long-term impact of such programs have shown the positive results they can have, including a reduction in <a href="https://www.dur.ac.uk/resources/criva/ProjectMirabalexecutivesummary.pdf">physical violence</a>, <a href="http://driveproject.org.uk/wp-content/uploads/2020/01/Drive-Evaluation-Report-Executive-Summary-Final.pdf">sexual violence and stalking behaviours</a>. This is why creating tools to change behaviour within domestic violence prevention interventions, like Choice-Point, are needed. </p>
<p>However, the <a href="https://www.gov.uk/government/publications/domestic-abuse-bill-2019-factsheets">Domestic Abuse Bill</a> (currently on-hold in Parliament) barely mentions perpetrators. If the bill included the use of prevention programmes, we could seek to meaningfully reduce the number of victim-survivors in the UK by challenging abusive behaviour at its source rather than after abuse reoccu.</p><img src="https://counter.theconversation.com/content/130556/count.gif" alt="The Conversation" width="1" height="1" />
<p class="fine-print"><em><span>Rosanna Bellini receives funding from the Engineering and Physical Sciences Research Council (EPSRC).
. </span></em></p>Through choose-your-own-adventure stories, perpetrators of domestic abuse can challenge and understand their behaviour.Rosanna Bellini, PhD Candidate in Digital Civics, Newcastle UniversityLicensed as Creative Commons – attribution, no derivatives.tag:theconversation.com,2011:article/1282202019-12-09T19:02:09Z2019-12-09T19:02:09ZHow our screen stories of the future went from flying cars to a darker version of now<figure><img src="https://images.theconversation.com/files/305795/original/file-20191209-90597-1u7c07j.jpg?ixlib=rb-1.1.0&rect=359%2C21%2C1688%2C1307&q=45&auto=format&w=496&fit=clip" /><figcaption><span class="caption">Years and Years begins with the re-election of Trump in the US, and the election of unconventional populist Four Star Party in the UK.</span> <span class="attribution"><span class="source">BBC/HBO</span></span></figcaption></figure><p>Fans of Ridley Scott’s 1982 masterpiece <a href="https://www.imdb.com/title/tt0083658/">Blade Runner</a> returned to cinemas last month for an unusual milestone: history catching up with science fiction.</p>
<p>Blade Runner opens in Los Angeles, in November 2019. Furnaces burst flames into the perennial night and endless rain. Flying cars zoom by. The antihero film-noir detective, Deckard (Harrison Ford) has seen too much, drinks too much, and misses his mother between “retiring replicants”.</p>
<p>As in “<a href="https://www.businessinsider.com.au/why-october-21-2015-was-the-chosen-date-in-back-to-the-future-part-ii-2015-10?r=US&IR=T">Back to the Future day</a>”, (October 21, 2015), which marked Marty McFly’s journey into the future in <a href="https://en.wikipedia.org/wiki/Back_to_the_Future_Part_II">the 1989 film</a>, the Blade Runner screenings came with a flurry of discussion about what the filmmakers <a href="https://www.theinquirer.net/inquirer/feature/3083388/blade-runner-november-2019-tech-accuracy">got right and wrong</a>. Environmental collapse, yes. But where are our flying cars?</p>
<figure>
<iframe width="440" height="260" src="https://www.youtube.com/embed/P1jXmJmmj3o?wmode=transparent&start=0" frameborder="0" allowfullscreen=""></iframe>
</figure>
<p>So: what now that the future is here? </p>
<p>Our current versions of near future stories - namely the television series <a href="https://www.imdb.com/title/tt2085059/?ref_=fn_al_tt_1">Black Mirror</a> (now on Netflix) and SBS’s <a href="https://www.imdb.com/title/tt8694364/?ref_=fn_al_tt_1">Years and Years</a> - explore more extreme versions of the present.</p>
<p>Charlie Brooker’s Black Mirror is an anthology of standalone episodes, produced between 2011 and 2019, each set in a slightly different, undated, near future. </p>
<p>Years and Years, written by Russell T. Davies, bravely spans 2019 to 2034 with each episode leaping forward a few years through striking montages of fictional news events: the collapse of the European Union, the US leaving the United Nations, catastrophic flooding, mass migration, widespread homelessness.</p>
<p>We are in a very familiar world. The “near” is depicted in a realistic way through identifiable locations, documentary-style visuals, news footage, and lifelike dialogue. </p>
<figure>
<iframe width="440" height="260" src="https://www.youtube.com/embed/lR5mZjgESIo?wmode=transparent&start=0" frameborder="0" allowfullscreen=""></iframe>
</figure>
<h2>Technology: good and bad</h2>
<p>Back in the real world, the future in the 21st century is unfolding in the palm of our hands. Elections are <a href="https://www.sbs.com.au/news/vote-2019-social-media-is-emerging-as-the-newest-political-battlefield">won and lost on social media</a>, Sydney is <a href="https://www.abc.net.au/news/2019-12-03/sydney-air-quality-smoke-haze-worse-this-bushfire-season/11755546">covered in smoke</a>. The rate at which technology is altering our lives is rivalled only by the rate we’re transforming our planet. </p>
<p>These shows explore these rates of change. In a 2016 episode of Black Mirror, “Nosedive”, every interpersonal interaction becomes a transaction: an extreme version of Uber Ratings with <a href="https://theconversation.com/chinas-social-credit-system-puts-its-people-under-pressure-to-be-model-citizens-89963">China’s Social Credit System</a>.</p>
<hr>
<p>
<em>
<strong>
Read more:
<a href="https://theconversation.com/chinas-social-credit-system-puts-its-people-under-pressure-to-be-model-citizens-89963">China’s Social Credit System puts its people under pressure to be model citizens</a>
</strong>
</em>
</p>
<hr>
<p>Lacie (Bryce Dallas Howard) is an ambitious young professional excited by the opportunities higher ratings open up, such as discounts on luxury apartments, but being pleasant to her barista and workmates only gets her so far. So begins a perilous spiral of trying too hard to be liked, echoing the personality-as-product phenomenon of social media influencers around the world. </p>
<figure>
<iframe width="440" height="260" src="https://www.youtube.com/embed/R32qWdOWrTo?wmode=transparent&start=0" frameborder="0" allowfullscreen=""></iframe>
</figure>
<p>The standalone episode format of Black Mirror means it can be challenging to develop empathy for characters, consequently the interest often rests on the single concept or final twist. The episode “Striking Vipers” explores the possibility of extra-marital love between best mates in Virtual Reality; “Hang the DJ” envisions dating apps as an authoritarian apparatus. </p>
<p>Most episodes are neatly wrapped up for viewers to escape to for pure entertainment – but also to escape from each dystopian possibility. </p>
<p>In Years and Years, we follow one Mancunian family over 19 years. The series opens with Trump re-elected for a second term. In the UK, the unconventional populist Four Star Party, led by straight-speaking Vivienne Rook (Emma Thompson), rides to success on the back of social instability. </p>
<p>Sci-fi concepts are introduced early on so we can explore their evolution and implications. In the first episode, teenager Bethany declares herself “trans”. As progressive parents, Stephen and Celeste immediately comfort their child, who they presume is transsexual. </p>
<figure>
<iframe width="440" height="260" src="https://www.youtube.com/embed/qOcktbXSfxU?wmode=transparent&start=0" frameborder="0" allowfullscreen=""></iframe>
</figure>
<p>Bethany shrugs, “I’m not transsexual … I’m transhuman”. A concept not lost on Blade Runner fans who may be aware of <a href="https://www.nytimes.com/2000/07/11/us/futurist-known-as-fm-2030-is-dead-at-69.html">transhumanist</a> gatherings in Los Angeles in the 1980s, transhumanism is premised on the idea that humans have breached evolutionary constraints through science and technology. Biology is a restriction to the possibility of eternal life. </p>
<hr>
<p>
<em>
<strong>
Read more:
<a href="https://theconversation.com/super-intelligence-and-eternal-life-transhumanisms-faithful-follow-it-blindly-into-a-future-for-the-elite-78538">Super-intelligence and eternal life: transhumanism's faithful follow it blindly into a future for the elite</a>
</strong>
</em>
</p>
<hr>
<p>Disgust and dismay ensue from parents unable to comprehend why their child wants to rid her flesh and live forever as data. Through the course of the series we see how Bethany’s transhuman ambitions influence her personal relationships, health, career trajectory, and political activism. </p>
<p>It even starts to feel normal.</p>
<p>Years and Years delicately resists portraying a dystopia, allowing room for technology to demonstrate a positive influence on society. “Señor”, the ubiquitous virtual assistant, connects the Lyons family whenever they wish. Like Alexa or Siri, Señor is always at hand to answer questions – but more importantly, facilitates an intimacy that could easily be lost to technological isolation.</p>
<p>In 2029, grandmother Muriel digs up the dusty digital assistant Señor because she misses its company. By now, virtual assistants are embedded into the walls and omnipresent digital cloud but the Luddite grandmother resists. </p>
<p>“I like having something to look at, I’m not talking to the walls like <a href="https://www.imdb.com/title/tt0098319/">Shirley Valentine</a>,” she says. </p>
<p>It’s moments like these that remind us of our agency over technology and hint at its revolutionary potential to connect us all. </p>
<h2>Lessons for the present</h2>
<p>While classics like Blade Runner looked to the future to ignite our technological desires, near-future fiction reveals how new technologies are injected into our lives with little choice as to whether we should adopt them and little thought to their long-term appropriateness and sustainability. </p>
<p>These shows ask us to be critical of what might seem like minor developments in technology and politics. In an age of rapidly changing political landscapes and the climate catastrophe, it can feel like we are approaching the final frontier. In creating stories set in the near, instead of the far, future, science fiction provides valuable lessons for the present. </p>
<p>In other words: the choices we fail to stand up for in the near-future may prevent us from having a distant future at all.</p><img src="https://counter.theconversation.com/content/128220/count.gif" alt="The Conversation" width="1" height="1" />
<p class="fine-print"><em><span>Aaron Burton does not work for, consult, own shares in or receive funding from any company or organisation that would benefit from this article, and has disclosed no relevant affiliations beyond their academic appointment.</span></em></p>Blade Runner’s vision of the future didn’t quite eventuate. Current TV shows such as Years and Years and Black Mirror explore more extreme versions of the present.Aaron Burton, Lecturer in Media Arts, University of WollongongLicensed as Creative Commons – attribution, no derivatives.tag:theconversation.com,2011:article/1182982019-06-04T21:47:51Z2019-06-04T21:47:51Z‘Black Mirror’: the dark side of technology<figure><img src="https://images.theconversation.com/files/277971/original/file-20190604-69091-ui1x2l.jpg?ixlib=rb-1.1.0&rect=0%2C51%2C3472%2C2229&q=45&auto=format&w=496&fit=clip" /><figcaption><span class="caption">Season 5 of _Black Mirror_ begins on June 5, 2019.
</span> <span class="attribution"><a class="source" href="http://www.allocine.fr/article/fichearticle_gen_carticle=18681311.html">Allocine</a></span></figcaption></figure><p>Because technology as its main character, <em>Black Mirror</em> is one of the most fascinating yet disturbing series of the last ten years. Fascinating, because viewers can easily identify with most situations. <a href="https://theinnersane.com/2019/05/05/these-haunting-episodes-of-black-mirror-are-still-giving-us-the-worst-nightmares/">Disturbing, because the technology is hostile</a>: addictive, invasive, spy, alienating, psychopathic, apocalyptic… Since Netflix broadcast <a href="https://www.youtube.com/watch?v=2bVik34nWws">the trailer for season 5</a> of <em>Black Mirror</em> on May 15, to launch on June 5, viewers have been eagerly awaiting the series, which, in very few episodes, has inspired the public, and many other shows.</p>
<figure>
<iframe width="440" height="260" src="https://www.youtube.com/embed/2bVik34nWws?wmode=transparent&start=0" frameborder="0" allowfullscreen=""></iframe>
<figcaption><span class="caption"><em>Black Mirror</em> season 5 trailer.</span></figcaption>
</figure>
<h2>An original concept for a powerful message</h2>
<p>Launched in 2011, <em>Black Mirror</em> not only aims to entertain, but it also invites us to think about how technology can <a href="https://theconversation.com/how-black-mirror-combines-a-disturbing-future-with-a-familiar-past-90659">harm society and transform our behaviour</a>. Each episode shows how an existing technology <a href="https://theconversation.com/how-black-mirror-combines-a-disturbing-future-with-a-familiar-past-90659">could evolve in the near future</a>, for better, or especially for worse. Technology can be dangerous in itself, but more often malicious designers or users use it to manipulate, humiliate, coerce, enslave or kill.</p>
<p>In <em>Black Mirror</em>, the situations are familiar, but are <a href="https://medium.com/@galencro/whats-up-with-car-accidents-and-black-mirror-season-4-24bcf967d5d7">pushed to the extreme</a>, provoking anxiety, destruction and even death. Each episode is independent, with its own universe and style, though clues <a href="https://screenrant.com/hints-black-mirror-shared-universe/">sometimes link them</a>. The topics covered include the obsession with celebrities, reality TV, <a href="https://www.mdpi.com/1660-4601/14/3/311">social networks</a>, <a href="https://www.childpsych.theclinics.com/article/S1056-4993(17)30139-6/abstract">video games</a>, <a href="https://www.ingentaconnect.com/content/prin/csj/2015/00000049/00000002/art00014">smartphones</a>, and <a href="https://www.tandfonline.com/doi/abs/10.1080/10720162.2019.1567411">pornography</a>; the end of private life; <a href="https://www.taylorfrancis.com/books/9781315591070">robots and androids</a>; social and commercial profiling; <a href="https://ora.ox.ac.uk/objects/uuid:cef7e8d9-27bf-4ea5-9fd6-855209b3e1f6">fake news and opinion manipulation</a>; dating sites and matching systems; <a href="https://link.springer.com/chapter/10.1007/978-3-319-90059-9_7">immersive augmented reality</a>; <a href="https://ieeexplore.ieee.org/abstract/document/8203709">cybersecurity and cyberbullying</a>; the transfer of memory or consciousness into a machine; and <a href="https://books.google.com/books?hl=en&lr=&id=FQSLCwAAQBAJ&oi=fnd&pg=PA1&dq=transhumanism+dangers&ots=IrtBCXlUqk&sig=JvzNz8gJBXzGvuvD23r7KvfTvpg">transhumanism</a>.</p>
<p>One of the most iconic <em>Black Mirror</em> episodes is season 1, episode 2, <a href="https://www.youtube.com/watch?v=MIVg1qgC_sk">“Fifteen Million Merits”</a>. It presents a world close to the <a href="https://books.google.com/books?hl=en&lr=&id=WoKGAgAAQBAJ&oi=fnd&pg=PP1&dq=herbert+marcuse&ots=UsUcVhPix2&sig=nmstbkqVC-6IxuZgcYu8bfF7CRc">description of philosopher Herbert Marcuse</a>, in which <a href="https://www.pdcnet.org/philnow/content/philnow_2013_0097_0042_0044">humanity is enslaved by mass media, advertising and industry</a>. The main character, Bing, spends his days pedalling an exercise bike in front of a television screen, like the whole of the middle class. He earns credits, called “merits”, to buy products or services. In this materialistic society, where <a href="https://gohighbrow.com/black-mirror-has-technology-corrupted-us-what-are-the-dangers-of-technology/">technology dominates and corrupts</a>, everyone is constantly being filmed. Reality TV and gaming are the only entertainment, and fame the only ambition. Not using pornography is a crime. Although Bing wants to revolt and denounce the system that oppresses him, he eventually becomes part of it, abandoning his morals for a comfortable life.</p>
<figure>
<iframe width="440" height="260" src="https://www.youtube.com/embed/MIVg1qgC_sk?wmode=transparent&start=0" frameborder="0" allowfullscreen=""></iframe>
<figcaption><span class="caption">The most powerful moments in <em>Black Mirror</em>, episode “Fifteen Million Merits”.</span></figcaption>
</figure>
<h2>From horrifying science fiction to tragic reality</h2>
<p>While <em>Black Mirror</em> seems to unfold in the near future, the show depicts <a href="https://digitalcommons.fiu.edu/classracecorporatepower/vol5/iss1/4/">worrying current trends</a>. Its creator, <a href="https://www.imdb.com/name/nm0111765/?ref_=nv_sr_1?ref_=nv_sr_1">Charlie Brooker</a>, surfs the technological news, cleverly incorporating it into <a href="https://theconversation.com/god-is-an-algorithm-why-were-closer-to-a-black-mirror-style-reality-than-we-think-90669">terrifying scenarios</a>. While these fantastic stories may <a href="https://identity-mag.com/black-mirror-exaggeration/">seem exaggerated</a>, technology, particularly digital technology, is <a href="https://www.taylorfrancis.com/books/9781315575667">increasingly used in crime</a>, is responsible for <a href="https://journals.sagepub.com/doi/abs/10.1177/0165551508095781">multiple pathologies</a>, and causes <a href="https://kops.uni-konstanz.de/bitstream/handle/123456789/30972/Weidmann_0-289398.pdf?sequence=1">geopolitical conflict</a>, the <a href="https://muse.jhu.edu/article/427159/summary">drift toward authoritarianism</a> and social deterioration. <em>Black Mirror</em> prophecies that <a href="https://www.thedailybeast.com/black-mirror-is-tvs-magic-8-ball">have come true</a> are the subject of heated debate.</p>
<p>The perverse behaviour presented in several episodes of <em>Black Mirror</em> can be observed on social networks. On May 15, a <a href="https://www.theguardian.com/world/2019/may/15/teenage-girl-kills-herself-after-instagram-poll-in-malaysia">16-year-old Malaysian teenager committed suicide</a> after asking her Instagram followers if she should live or die, because 69% told her to end her life. Facebook has also been blamed for broadcasting live 17 minutes of the <a href="http://time.com/5589478/facebook-tightens-live-stream-rules-in-response-to-the-christchurch-massacre/">March 15 attacks in Christchurch, New Zealand</a> in which 51 people died. <em>Black Mirror</em> repeatedly illustrates the morbid, sordid tastes of many anonymous online viewers, as in <a href="https://www.youtube.com/watch?v=JiFE84i1N_c">the very first episode</a>.</p>
<figure>
<iframe width="440" height="260" src="https://www.youtube.com/embed/JiFE84i1N_c?wmode=transparent&start=0" frameborder="0" allowfullscreen=""></iframe>
<figcaption><span class="caption"><em>Black Mirror</em>, trailer for the “National Anthem” episode.</span></figcaption>
</figure>
<h2>A futuristic show that questions the present</h2>
<p>Season 3, episode 1, <a href="https://www.youtube.com/watch?v=epI5r0_T_lE">“Nosedive”</a> describes an oppressive society where everyone constantly grades other people’s words, actions, and publications on a five-point scale. They are equipped with eye implants enabling them to see the grades of those around them. This satire on a society where everyone is <a href="https://www.youtube.com/watch?v=R32qWdOWrTo">obsessed by their own image</a> criticises superficiality and the need to portray oneself positively to obtain other people’s approval. The episode illustrates perfectly the consequences of <a href="https://theconversation.com/chinas-social-credit-system-puts-its-people-under-pressure-to-be-model-citizens-89963">China’s “social credit” system</a>, which monitors and records the entire population in real time to “improve” behaviour and strengthen the regime.</p>
<p>In season 3, episode 2, <a href="https://www.youtube.com/watch?v=RUM_evdCl4I">“Playtest”</a>, a gamer tries an experimental video game that kills him in a few milliseconds. The guinea pig is equipped with a virtual reality headset and a neural implant. The implant <a href="https://ora.ox.ac.uk/objects/uuid:426d61fb-deab-411c-8778-ca7271e53ead">hacks his brain</a> and generates images based on his fears and physiological data. The episode warns against <a href="https://www.theverge.com/2016/10/25/13401020/black-mirror-season-3-episode-2-playtest-recap">giving developers control of human senses</a>, and about the immersive, invasive technologies that are the <a href="https://www.globalme.net/blog/black-mirror-virtual-reality-playtest">future of video games</a>. “Playtest” was first broadcast on Netflix on October 21, 2016, a few months after the release of the <a href="https://www.oculus.com/">Facebook Oculus Rift Headset</a> on March 28, 2016.</p>
<p>Season 4, episode 5, <a href="https://www.youtube.com/watch?v=xejjA2AFO5I">“Metalhead”</a>, describes a post-apocalyptic world, where intelligent killer dogs hunt humans. This is a direct reference to the company <a href="http://nymag.com/intelligencer/2017/06/google-sells-boston-dynamics-to-softbank.html/">Boston Dynamics</a>, which Google sold in 2017 to the Japanese group Softbank. In the series, these dogbots appear on <a href="https://www.youtube.com/watch?v=wND9goxDVrY">building sites and in public</a>, but the project was initially a <a href="https://sputniknews.com/science/201811181069917338-autonomous-al-threat-of-killer-robots/">partnership with DARPA</a> (Defense Advanced Research Projects Agency), a US military R&D agency.</p>
<figure>
<iframe width="440" height="260" src="https://www.youtube.com/embed/9EcY6VDgz1M?wmode=transparent&start=0" frameborder="0" allowfullscreen=""></iframe>
<figcaption><span class="caption">Fiction: special effects in the <em>Black Mirror</em> episode “Metalhead”.</span></figcaption>
</figure>
<figure>
<iframe width="440" height="260" src="https://www.youtube.com/embed/wND9goxDVrY?wmode=transparent&start=0" frameborder="0" allowfullscreen=""></iframe>
<figcaption><span class="caption">Reality: Boston Dynamics robot testing at construction sites.</span></figcaption>
</figure>
<h2>Imitated but never equalled</h2>
<p>The Season 5 promotional campaign reminds us that the series has changed its viewers’ perceptions of the world and has received multiple prizes, including <a href="https://www.independent.co.uk/arts-entertainment/tv/news/emmys-2017-winners-black-mirror-san-junipero-charlie-brooker-outstanding-writing-tv-movie-a7952266.html">Emmys</a> and BAFTA Awards. The reviews are excellent, with an overall score of <a href="https://www.imdb.com/title/tt2085059/ratings?ref_=tt_ov_rt">8.9/10 on IMDb</a>.</p>
<p>Similar shows have tried to imitate the success, such as the miniseries <a href="https://www.youtube.com/watch?v=rvwS1bo0fu4"><em>Electric Dreams</em></a>, based on sci-fi author Philip K. Dick’s short novels, many of which have been <a href="https://www.senscritique.com/top/resultats/Les_best_adaptations_de_Philip_K_Dick/1712116">adapted for the screen</a>. Considered a <a href="http://dangerousuniverse.com/du/tag/philip-k-dicks-electric-dreams/">clone of <em>Black Mirror</em></a>, <em>Electric Dreams</em> was first aired on Channel 4 in Britain, like its model. Amazon Video acquired the US rights, and with <a href="https://www.imdb.com/name/nm0186505/?ref_=nv_sr_1?ref_=nv_sr_1">Bryan Cranston</a> as executive producer and an <a href="https://www.express.co.uk/showbiz/tv-radio/923855/Electric-Dreams-cast-Philip-K-Dick-Channel-4-Amazon-Bryan-Cranston-Richard-Madden">all-star cast</a>, the series was successful, but did not attain the cult status of <em>Black Mirror</em>.</p>
<p><a href="https://www.youtube.com/watch?v=9BqKiZhEFFw"><em>Westworld</em></a> also seems strongly influenced by <em>Black Mirror</em>. Its first two seasons focused on robotics and transhumanism. HBO has just released the <a href="https://www.youtube.com/watch?v=deSUQ7mZfWk">trailer for the third season</a>, planned for 2020, in which a new character, played by <a href="https://www.imdb.com/name/nm0666739/?ref_=nv_sr_1?ref_=nv_sr_1">Aaron Paul</a> (who played Jesse Pinkman in <a href="https://www.youtube.com/watch?v=HhesaQXLuRY"><em>Breaking Bad</em></a>, and who <a href="https://www.youtube.com/watch?v=RffKB_xJojo">performed a voiceover in an episode of <em>Black Mirror</em></a>) condemns the false promise of a better world made to justify the technological development and commercialisation of servo devices.</p>
<p><em>Black Mirror</em> has often been compared to <a href="https://www.imdb.com/title/tt0052520/?ref_=nv_sr_2?ref_=nv_sr_2"><em>The Twilight Zone</em></a> in the 1960s, and is probably at the origin of its <a href="https://www.imdb.com/title/tt2583620/?ref_=nv_sr_1?ref_=nv_sr_1">reboot</a>, although the showrunner <a href="https://www.imdb.com/name/nm1443502/?ref_=nv_sr_1?ref_=nv_sr_1">Jordan Peele</a> – author, director and producer of the thrillers <a href="https://www.youtube.com/watch?v=sRfnevzM9kQ"><em>Get Out</em></a> and <a href="https://www.youtube.com/watch?v=hNCmb-4oXJA"><em>Us</em></a> – denies the similarity. Jordan Peele is also co-author of the series <a href="https://www.youtube.com/watch?v=raJJbbiKtlY"><em>Weird City</em></a>, released on February 13, 2019, on YouTube Premium. Its <a href="https://heavy.com/entertainment/2019/02/weird-city-episode-1-cast-special-guests/">well-chosen guest-stars</a> present the dangers of technology with more humour.</p>
<p>AMC, which broadcast popular series like <em>Mad Men</em>, <em>Breaking Bad</em> and <em>The Walking Dead</em>, has <a href="https://variety.com/2019/tv/news/amc-orders-sc-fi-anthology-series-from-black-mirror-writer-1203217531/">commissioned a new series for 2020 from Will Bridges</a>, one of the authors of <em>Black Mirror</em>, that will focus on a dating technology that finds your soul mate. Since the original is often better than its copies, let’s hope that Netflix will produce many more seasons of <em>Black Mirror</em>.</p><img src="https://counter.theconversation.com/content/118298/count.gif" alt="The Conversation" width="1" height="1" />
<p class="fine-print"><em><span>Oihab Allal-Chérif ne travaille pas, ne conseille pas, ne possède pas de parts, ne reçoit pas de fonds d'une organisation qui pourrait tirer profit de cet article, et n'a déclaré aucune autre affiliation que son organisme de recherche.</span></em></p>If “Black Mirror” is one of the most fascinating and disturbing series of the last ten years, it is because of its main character: technology.Oihab Allal-Chérif, Full Professor, Information Systems, Purchasing and Supply Chain Management, Neoma Business SchoolLicensed as Creative Commons – attribution, no derivatives.tag:theconversation.com,2011:article/1152732019-04-22T10:45:37Z2019-04-22T10:45:37ZWill Netflix eventually monetize its user data?<figure><img src="https://images.theconversation.com/files/269022/original/file-20190412-76856-y27th8.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=496&fit=clip" /><figcaption><span class="caption">Netflix currently spends much more cash than it brings in, leading to consistent negative cash flow and a mountain of debt.</span> <span class="attribution"><a class="source" href="https://www.shutterstock.com/image-photo/us-president-george-washington-face-portrait-1020235492">sakhorn/Shutterstock.com</a></span></figcaption></figure><p>Even in the wake of a <a href="https://finance.yahoo.com/news/netflix-earnings-q1-2019-183928130.html">recent mixed earning report and volatile stock prices</a>, Netflix remains the media success story of the decade. The company, whose user base has <a href="http://infographic.statista.com/normal/chartoftheday_10311_netflix_subscriptions_usa_international_n.jpg">grown rapidly</a>, now boasts almost 150 million global subscribers.</p>
<p><a href="https://scholar.google.com/citations?user=I2Zb3TsAAAAJ&hl=en">But as someone who studies the television industry</a>, I’ve always wondered how Netflix can provide so much unlimited ad-free content for such a low monthly rate, which currently averages around US$14. </p>
<p>After all, didn’t <a href="https://www.theringer.com/movies/2019/2/6/18212482/moviepass-mitch-lowe-khalid-itum-interview-2019">MoviePass just fall apart</a> using a similar model of offering ad-free content for a monthly subscription fee? And Netflix is <a href="https://www.cnn.com/2019/01/18/investing/netflix-cash-burn-stock/index.html">burning through cash</a>, with negative cash flow of $3 billion in 2018 alone. </p>
<p>What if we’re looking at Netflix through the wrong lens? What if its primary long-term business model is not as a media content or distribution company, but as a data aggregation company? </p>
<p>Seeing Netflix this way might better explain its current strategy and clue us into the company’s future plans, while <a href="https://www.nytimes.com/interactive/2019/opinion/internet-privacy-project.html">raising red flags about ethics and privacy</a>.</p>
<h2>Spending more and charging less</h2>
<p>For a century of screen entertainment, there were only a few ways for Americans to pay for media:</p>
<ul>
<li><p>You could purchase a book, album or DVD, “lease” a movie theater seat or rent a tape at a video store;</p></li>
<li><p>You could pay with your attention by consuming ads alongside “free” radio or television programming;</p></li>
<li><p>Or you could subscribe to cable TV, and pay a large monthly fee to access an array of scheduled programming.</p></li>
</ul>
<p>Netflix doesn’t follow any of these three models. Instead it most resembles HBO’s subscription service, which similarly provides ad-free original programming alongside a library of older content for a monthly fee. </p>
<p>While they may seem analogous, there are key differences. HBO is part of a larger media company, which gives it access to vast content libraries. And even though HBO charges more than Netflix, it spends far less for original content. In 2017, <a href="http://fortune.com/2018/07/08/netflix-original-programming-13-billion/">HBO spent $2.5 billion to Netflix’s $8 billion</a>. The latter’s spending grew to $13 billion in 2018.</p>
<h2>Relying on subscribers, not ads</h2>
<p>Pouring money into content might generate hits, but not direct profits: Netflix’s sole revenue stream is subscriptions, so its primary goal is to gain and retain subscribers. Having <a href="https://www.theatlantic.com/entertainment/archive/2016/02/netflix-ratings/462447/">popular content generates buzz</a>, and Netflix hypes its brand by using <a href="https://www.vox.com/culture/2019/2/6/18212781/netflix-viewership-explained">self-reported numbers</a> to claim that its original films and series like “Bird Box” and “Sex Education” attract millions of viewers. Yet Netflix only yields the same monthly fee per household, regardless of how much subscribers watch. </p>
<p>This makes Netflix distinct from other media companies that use highly profitable hits to generate revenue. This will then subsidize the production of new films, television shows, albums and video games. </p>
<p>Meanwhile, competing streaming platforms Hulu and Amazon Prime Video have other revenue sources – <a href="https://www.businessinsider.com/hulu-netflix-secret-weapon-2017-6">advertising</a> and <a href="https://variety.com/2018/digital/news/amazon-prime-video-channels-tv-revenue-estimates-1203083998/">retail</a>, respectively – and their larger diversified companies can better leverage hits.</p>
<p>Netflix needs to produce and acquire desirable content to make the service indispensable. But making original content is expensive. Hiring talent and producing movies and television series costs the company <a href="https://variety.com/2019/digital/news/netflix-content-spending-2019-15-billion-1203112090/">more than $15 billion annually</a>. Netflix spends much more cash than it brings in, leading to consistent negative cash flow and a <a href="https://finance.yahoo.com/news/netflix-earnings-q1-2019-183928130.html">mountain of debt that amounts to more than $10 billion</a>.</p>
<p>Even though <a href="https://www.netflixinvestor.com/financials/quarterly-earnings/default.aspx">it reported a record $1.2 billion in profit in 2018</a>, those profits are based on an accounting model that ignores many costs and debts. This has led some financial analysts, like <a href="https://www.nytimes.com/2018/10/26/business/netflix-audience-stock-debt.html">NYU professor Aswath Damodaran</a>, to believe that Netflix’s business model is unsustainable. </p>
<p>“The more Netflix grows,” he wrote last fall, “the more its costs grow and the more money it burns. I’m not sure how it’s ever going to turn that around.” </p>
<p>So with only one stable, inflexible revenue source, how might Netflix’s business model become more sustainable? </p>
<h2>More analogous to Facebook?</h2>
<p>One theory is that Netflix is playing the long game, pitting itself against social media companies like Facebook and YouTube, rather than just film studios or TV networks.</p>
<p>Media commentator <a href="https://redef.com/original/5b400a2779328f4711d5675e">Matthew Ball</a> argues that Netflix is in a race with the social media giants to occupy “every minute of leisure time available.”</p>
<p>Yet Netflix’s financial model is the inverse of Facebook’s and YouTube’s. The social media giants generate <a href="http://www.businessofapps.com/data/youtube-statistics/">huge advertising revenues from free, user-generated content</a>. Perhaps Netflix could balance content costs with higher subscription fees and its growing global user base. It seems unlikely, however, that this model could lead to anything beyond small profit margins.</p>
<p>But what if the parallel between Netflix and Facebook runs deeper than cost and revenue?</p>
<p>From its inception as a DVD rental service, <a href="https://www.buzzfeednews.com/article/nicolenguyen/netflix-recommendation-algorithm-explained-binge-watching">Netflix has touted its competitive advantage through its algorithm</a> – the predictive engine that claims to deliver the most user-specific content from its vast library. Netflix has always been a technology firm first and foremost, invested in mining its library of vast user data to deliver what viewers want to watch.</p>
<p>For instance, the <a href="https://www.wsj.com/articles/at-netflix-who-wins-when-its-hollywood-vs-the-algorithm-1541826015">Netflix engineering team strives</a> “to have customers click on a show in the first 10 seconds.” Such obsessive interface tweaking helps promote programming – <a href="https://finance.yahoo.com/news/why-netflix-winning-2019-bullish-160207852.html">as Ball notes</a>, “the most valuable real estate in the world is the top fold of Netflix home page.” But it doesn’t generate revenue.</p>
<p>This emphasis on viewing optimization, internal promotion and maximizing engagement resonates with another recent Netflix offering: the “Black Mirror” episode “<a href="https://www.hollywoodreporter.com/live-feed/black-mirror-bandersnatch-endings-explained-1171556">Bandersnatch</a>.” Netflix’s highest-profile experiment in interactive narrative, “Bandersnatch” allows viewers to choose how the story unfolds from dozens of options.</p>
<p>Netflix collects data from viewers of “Bandersnatch,” charting the narrative choices they made during the episode. Such viewer activity feeds into Netflix’s tracking efforts that it uses to make programming decisions and customize promotion to each subscriber. </p>
<p>A logical next step would be product integration. Based on your choices within the narrative around specific brand names, <a href="https://www.theverge.com/2019/1/2/18165182/black-mirror-bandersnatch-netflix-interactive-strategy-marketing">Netflix could then sell customized micro-targeted product placements</a> within programs – a strategy that could actually lead to increased revenue.</p>
<h2>A data gold mine?</h2>
<p>Based on all we know about <a href="https://www.theguardian.com/technology/2016/aug/31/personal-data-corporate-use-google-amazon">Silicon Valley’s aggressive monetization of user data</a>, what else could Netflix do, beyond product integration, with this valuable information?</p>
<p>Netflix logs everything you have ever watched and how you watch – every time you pause, what programs you consider watching but choose not to and when you’re most likely to binge on “Friends” reruns. </p>
<p>When linked to website trackers, Netflix could, for example, cross-reference that viewing data with your social media accounts, your purchasing habits, your search history and even your emails. </p>
<p>In the <a href="https://www.nytimes.com/2019/04/10/opinion/sunday/privacy-capitalism.html">age of surveillance capitalism</a>, such data could be worth a fortune to marketers, political campaigns and advertisers.</p>
<p>As far as we know, Netflix has not started using its data to track us online, package us to marketers or cross-reference our private messages (even though <a href="https://www.theverge.com/2018/12/18/18147616/facebook-user-data-giveaway-nyt-apple-amazon-spotify-netflix">Facebook has provided Netflix access to this information</a>). And I doubt Netflix will violate its core brand by incorporating ads into its interface. Partnering with or acquiring a marketing firm to suffuse every subscriber’s online experiences with micro-targeted ads seems more likely. </p>
<p>All of these potential uses of viewing data are still speculative. But since profits regularly eclipse tech companies’ ethical standards, it’s important to be asking these questions before, rather than after, the damage is done.</p><img src="https://counter.theconversation.com/content/115273/count.gif" alt="The Conversation" width="1" height="1" />
<p class="fine-print"><em><span>Jason Mittell does not work for, consult, own shares in or receive funding from any company or organization that would benefit from this article, and has disclosed no relevant affiliations beyond their academic appointment.</span></em></p>Something about Netflix’s business model just doesn’t add up – unless you look at the streaming service as a massive data collection company.Jason Mittell, Professor of Film & Media Culture, MiddleburyLicensed as Creative Commons – attribution, no derivatives.tag:theconversation.com,2011:article/1110372019-03-27T10:31:14Z2019-03-27T10:31:14ZBeyond ‘Bandersnatch,’ the future of interactive TV is bright<figure><img src="https://images.theconversation.com/files/263261/original/file-20190311-86713-1yqd5ty.jpg?ixlib=rb-1.1.0&rect=0%2C66%2C1897%2C985&q=45&auto=format&w=496&fit=clip" /><figcaption><span class="caption">Make a choice to see the next phase of the story.</span> <span class="attribution"><a class="source" href="https://media.netflix.com/en/only-on-netflix/341557">Netflix</a></span></figcaption></figure><p>Make a choice: Do you want to engage with your media passively or actively?</p>
<p>The December 2018 premiere of Netflix’s “Black Mirror: Bandersnatch” offered consumers a new way to influence the entertainment they’re watching. <a href="https://help.netflix.com/en/node/62526">Netflix has a growing list</a> of choose-your-own-adventure movies. What viewers might see as a simple choice, such as which breakfast cereal a character begins the day with, could affect the whole show’s storyline. There are other choices to make as well – some of which change the plot, and some of which may not.</p>
<p>Viewers aren’t watching these interactive films just once. Rather, they are <a href="https://mlatgt.blog/2019/01/03/machine-learning-meets-interactive-stories/">watching them over and over again</a> to find each ending and <a href="https://www.ign.com/wikis/black-mirror/Bandersnatch_Endings">post maps of the diverging plot lines</a>. I think I sat on my couch for nearly three hours straight trying to <a href="https://www.thewrap.com/black-mirror-bandersnatch-netflix-5-main-endings-secret-kill-dad-movie-set-mom-pearl-ritman/">exhaust all of “Bandersnatch’s” choices</a> as it followed a programmer and designer through the process of game development.</p>
<p>I’ve been teaching and researching <a href="https://scholar.google.com/citations?user=C7XXFqMAAAAJ&hl=en">game design and development</a> since 2001. I see this type of experience not as just the future of entertainment, but as the expansion of a standard method of storytelling that <a href="https://forensic-games.csec.rit.edu/">game designers have been using</a> for decades. Netflix is introducing new technology and new audiences to this type of entertainment, but fiction writers have been exploring similar themes for far longer, creating stories of time travel and alternative realities that let people fantasize about redoing decisions in life.</p>
<h2>Controlling your own destiny</h2>
<p>There is a kind of game made popular by “<a href="http://dnd.wizards.com/">Dungeons & Dragons</a>” that provides a way to understand and expand what “Bandersnatch” explores. Role-playing games let players pick characters with multiple traits, such as strength, health and special skills, and work together to achieve story-driven goals. </p>
<p>Fans of “<a href="https://www.tolkiensociety.org/">The Lord of the Rings</a>” books and movies will recognize the idea of a <a href="https://lotr.fandom.com/wiki/The_Fellowship_of_the_Ring_(group)">team of characters with different backgrounds</a>, abilities and motivations, all trying to work together toward a goal. The adventure is not just in whether they achieve the task, but the encounters, mishaps and even battles that happen along the way. The ultimate outcome depends on the choices players make along the way.</p>
<figure class="align-center zoomable">
<a href="https://images.theconversation.com/files/263679/original/file-20190313-123545-14ybvhk.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=1000&fit=clip"><img alt="" src="https://images.theconversation.com/files/263679/original/file-20190313-123545-14ybvhk.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=754&fit=clip" srcset="https://images.theconversation.com/files/263679/original/file-20190313-123545-14ybvhk.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=600&h=402&fit=crop&dpr=1 600w, https://images.theconversation.com/files/263679/original/file-20190313-123545-14ybvhk.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=600&h=402&fit=crop&dpr=2 1200w, https://images.theconversation.com/files/263679/original/file-20190313-123545-14ybvhk.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=600&h=402&fit=crop&dpr=3 1800w, https://images.theconversation.com/files/263679/original/file-20190313-123545-14ybvhk.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=754&h=505&fit=crop&dpr=1 754w, https://images.theconversation.com/files/263679/original/file-20190313-123545-14ybvhk.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=754&h=505&fit=crop&dpr=2 1508w, https://images.theconversation.com/files/263679/original/file-20190313-123545-14ybvhk.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=754&h=505&fit=crop&dpr=3 2262w" sizes="(min-width: 1466px) 754px, (max-width: 599px) 100vw, (min-width: 600px) 600px, 237px"></a>
<figcaption>
<span class="caption">Many role-playing games get people together around a computer to explore a collective adventure.</span>
<span class="attribution"><a class="source" href="http://www.apimages.com/metadata/Index/Business-of-Life/417a3174c8dd44da9f3269c0ec266081/11/0">AP Photo/Ted S. Warren</a></span>
</figcaption>
</figure>
<p>Role-playing games started with players gathered around a table, keeping notes on paper and rolling dice to incorporate the role of chance and probability into the adventure. A human game master coordinated everything, keeping track of what was happening and working with players to advance their stories and the overall plot of the adventure. </p>
<p>Early computer games, such as the 1980s-era <a href="http://www.infocom-if.org/games/games.html">Infocom</a> <a href="https://classicreload.com/the-lost-treasures-of-infocom-volume-i.html">text adventures</a>, turned the role of game master over to a game designer, who controlled the choices and their consequences. In the decades since, more powerful computers have let modern digital games offer a great many choices. Teachers have begun to use <a href="https://doi.org/10.1145/1536513.1536526">elements of role-playing games</a> to help students learn.</p>
<h2>Illusion of choice</h2>
<p>With “Bandersnatch,” Netflix used software to process viewers’ choices and deliver the appropriate video. When watching and “playing,” I wondered if there were too few choices. The show offered only two choices of breakfast cereal, and the viewer couldn’t choose to skip breakfast, make eggs or open the freezer to grab some ice cream. But, there’s a very good reason for these constraints.</p>
<figure class="align-right zoomable">
<a href="https://images.theconversation.com/files/263685/original/file-20190313-123531-19debn0.png?ixlib=rb-1.1.0&q=45&auto=format&w=1000&fit=clip"><img alt="" src="https://images.theconversation.com/files/263685/original/file-20190313-123531-19debn0.png?ixlib=rb-1.1.0&q=45&auto=format&w=237&fit=clip" srcset="https://images.theconversation.com/files/263685/original/file-20190313-123531-19debn0.png?ixlib=rb-1.1.0&q=45&auto=format&w=600&h=449&fit=crop&dpr=1 600w, https://images.theconversation.com/files/263685/original/file-20190313-123531-19debn0.png?ixlib=rb-1.1.0&q=30&auto=format&w=600&h=449&fit=crop&dpr=2 1200w, https://images.theconversation.com/files/263685/original/file-20190313-123531-19debn0.png?ixlib=rb-1.1.0&q=15&auto=format&w=600&h=449&fit=crop&dpr=3 1800w, https://images.theconversation.com/files/263685/original/file-20190313-123531-19debn0.png?ixlib=rb-1.1.0&q=45&auto=format&w=754&h=564&fit=crop&dpr=1 754w, https://images.theconversation.com/files/263685/original/file-20190313-123531-19debn0.png?ixlib=rb-1.1.0&q=30&auto=format&w=754&h=564&fit=crop&dpr=2 1508w, https://images.theconversation.com/files/263685/original/file-20190313-123531-19debn0.png?ixlib=rb-1.1.0&q=15&auto=format&w=754&h=564&fit=crop&dpr=3 2262w" sizes="(min-width: 1466px) 754px, (max-width: 599px) 100vw, (min-width: 600px) 600px, 237px"></a>
<figcaption>
<span class="caption">Every story decision requires more writing and more development.</span>
<span class="attribution"><a class="source" href="https://www.flickr.com/photos/psychemedia/2383101962">Tony Hirst/Flickr</a>, <a class="license" href="http://creativecommons.org/licenses/by/4.0/">CC BY</a></span>
</figcaption>
</figure>
<p>I often tell my students that when they’re creating role-playing games, the problem isn’t giving players choices: It’s deciding what happens next. Giving players lots of options is great, and fun – but with every choice the job gets harder. If there are three kinds of ice cream in the freezer, that’s three different sets of video to show vanilla, chocolate and strawberry – and possibly three different scripts, if the choice actually has consequences. </p>
<p>In game design, we call this a “<a href="https://thestoryelement.wordpress.com/2015/02/11/designing-branching-narrative/">branching narrative</a>,” where every choice spawns as many new branches as there are options, and the tree gets bigger and bigger all the time. A movie with an enormous number of options would require multiple sets, extra time for actors, huge amounts of special effects work, extended production times and increasing budgets. </p>
<p>Such a complex film would also take viewers huge amounts of time to experience. Digital game players can handle this sort of effort by saving their progress and taking a break, returning to resume play hours later, or even days. </p>
<p>With an interactive movie, would a viewer want several days’ worth of watching? I don’t know if anyone has an idea of how long a typical interactive movie experience should last. My three hours on the couch watching “Bandersnatch” seemed about right – and ran through most of the options. </p>
<p>The Netflix producers borrowed from game designers, and the classic “<a href="https://www.cyoa.com/">Choose Your Own Adventure</a>” book series, to give viewers the illusion of choices when really the alternatives were limited. My own research recommended the same technique: <a href="https://doi.org/10.1109/ISECon.2015.7119899">Allow the players some choices</a>, but bring them back to the main narrative thread at key points.</p>
<h2>Future of interactive media</h2>
<p>There will be more interactive movies. Netflix has <a href="https://www.vulture.com/2018/12/netflix-created-wild-software-for-black-mirror-bandersnatch.html">built its own software</a> for “Bandersnatch,” which it can use <a href="https://arstechnica.com/gaming/2019/03/beyond-bandersnatch-netflix-plans-to-double-down-on-interactive-tv/">for other stories</a> too. There are already several addictive <a href="https://help.netflix.com/en/node/62526">interactive kids’ shows</a>, including “Puss in Book: Trapped in an Epic Tale,” “Buddy Thunderstruck: The Maybe Pile” and “Stretch Armstrong: The Breakout.”</p>
<p>Gamers are already familiar with this convergence of film, interactivity and branching narrative. Cinematic video games, like “Indigo Prophecy” and “<a href="http://www.quanticdream.com/en/#!/en/category/heavy-rain">Heavy Rain</a>,” let players make choices in dialog and other cinematic aspects, all of which alter the endings. An academically published game, “<a href="https://www.playablstudios.com/facade">Façade</a>,” is considered important not just for showing that scholarly games can be fun to play, but also demonstrating that academic concepts of branching narrative and story can create meaningful play: The <a href="https://theconversation.com/the-future-is-in-interactive-storytelling-76772">player visits a couple’s apartment</a>, and depending on where the player moves and what the player says, the couple reacts in different ways.</p>
<figure class="align-right zoomable">
<a href="https://images.theconversation.com/files/167219/original/file-20170428-12999-1e203kp.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=1000&fit=clip"><img alt="" src="https://images.theconversation.com/files/167219/original/file-20170428-12999-1e203kp.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=237&fit=clip" srcset="https://images.theconversation.com/files/167219/original/file-20170428-12999-1e203kp.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=600&h=440&fit=crop&dpr=1 600w, https://images.theconversation.com/files/167219/original/file-20170428-12999-1e203kp.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=600&h=440&fit=crop&dpr=2 1200w, https://images.theconversation.com/files/167219/original/file-20170428-12999-1e203kp.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=600&h=440&fit=crop&dpr=3 1800w, https://images.theconversation.com/files/167219/original/file-20170428-12999-1e203kp.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=754&h=553&fit=crop&dpr=1 754w, https://images.theconversation.com/files/167219/original/file-20170428-12999-1e203kp.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=754&h=553&fit=crop&dpr=2 1508w, https://images.theconversation.com/files/167219/original/file-20170428-12999-1e203kp.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=754&h=553&fit=crop&dpr=3 2262w" sizes="(min-width: 1466px) 754px, (max-width: 599px) 100vw, (min-width: 600px) 600px, 237px"></a>
<figcaption>
<span class="caption">Engaging with a couple on the rocks.</span>
<span class="attribution"><a class="source" href="http://www.interactivestory.net/screenshot4.html">'Façade,' by Michael Mateas and Andrew Stern.</a>, <a class="license" href="http://creativecommons.org/licenses/by-nd/4.0/">CC BY-ND</a></span>
</figcaption>
</figure>
<p>I anticipate different genres of shows will explore interactive formats. Imagine playing through historical fiction where you can choose to <a href="https://www.history.com/topics/france/marie-antoinette">execute Marie Antoinette</a> or not. I also expect viewers will be able to make their choices in different ways than just pressing buttons on their remotes – perhaps by using voice recognition on their phones. </p>
<p>If artificial intelligence and machine learning systems get better at telling stories, viewers might even be able to suggest new possible choices, with the resulting content generated on the fly while people watch. Of course, there’s a strong overlap with virtual reality, offering <a href="https://mitpress.mit.edu/books/hamlet-holodeck-updated-edition">immersive escapism</a>, which is, in my experience, a key goal of interactivity.</p>
<p>In the meantime, “Bandersnatch” fans who want to continue exploring choosing their own adventures to direct a story can look for local gaming groups and game stores. “<a href="http://dnd.wizards.com/">Dungeons & Dragons</a>” and “<a href="https://www.kenzerco.com/hackmaster/">HackMaster</a>” are regaining popularity lately. So is <a href="https://larping.org/">live-action role-playing</a>, in which people physically act out their fictional encounters. In these environments, players can ask “what if” without running into the limitations of software development and movie production teams. Human players can engage in the full extent of their imagination without any illusion of choice.</p>
<iframe src="https://content.jwplatform.com/players/m4QKaKkH-puACk8ZV.html" width="100%" height="340" frameborder="0" scrolling="auto"></iframe>
<p><a href="https://www.businessinsider.com/what-is-larping-live-action-role-playing-steampunk-aurum-larp-2018-10">Live-action role playing</a>.</p><img src="https://counter.theconversation.com/content/111037/count.gif" alt="The Conversation" width="1" height="1" />
<p class="fine-print"><em><span>David Schwartz receives funding from the National Science Foundation (nsf.gov).</span></em></p>As Netflix plans additional choose-your-own-adventure TV movies, a game designer explains how they’re made and the long history of audience-directed fiction.David I. Schwartz, Associate Professor and Director, School of Interactive Games and Media, Rochester Institute of TechnologyLicensed as Creative Commons – attribution, no derivatives.tag:theconversation.com,2011:article/1098302019-01-17T11:08:50Z2019-01-17T11:08:50ZBandersnatch is just the start – the next big thing in interactive media is AI storytelling<figure><img src="https://images.theconversation.com/files/254074/original/file-20190116-163280-20kzuh.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=496&fit=clip" /><figcaption><span class="caption">
</span> <span class="attribution"><span class="source">Image courtesy of Netflix</span></span></figcaption></figure><p>The fifth Black Mirror season – made up of a single episode called “Bandersnatch” – represents the latest mainstream offering in interactive storytelling. Released in December 2018 by Netflix in a choose-your-own-adventure format, Bandersnatch allows viewers to make decisions at various junctures – these choices then determine the story path down which the episode proceeds. </p>
<p>The result is what has been described as a network of “<a href="https://www.indiewire.com/2018/12/black-mirror-bandersnatch-endings-one-trillion-story-combinations-netflix-streaming-1202031075/">five endings and one trillion story combos</a>”, including, apparently, some scenes that <a href="https://www.digitalspy.com/tv/a25897470/black-mirror-bandersnatch-why-deleted-scenes-impossible-find/">nobody can find</a>. The Independent described this storytelling format as “<a href="https://www.indiewire.com/2018/12/black-mirror-bandersnatch-review-groundbreaking-interactive-fun-netflix-spoilers-1202031049/">groundbreaking</a>” while The Guardian proclaimed that the “<a href="https://www.theguardian.com/tv-and-radio/2019/jan/01/black-mirror-bandersnatch-review-charlie-brooker-netflix-tv-of-tomorrow-is-now-here">TV of tomorrow is now here</a>”. On Twitter, fans gushed about its “amazing” storyline and how #Bandersnatch is “<a href="https://www.dailymail.co.uk/tvshowbiz/article-6537163/Black-Mirrors-Bandersnatch-drives-fans-wild-amazing-storyline.html">a genre-defining piece of art</a>”. </p>
<p>But the format is actually not a particularly new idea: recent (and not so recent, as we will see) history is littered with precedents. Netflix itself presented similar interactive episodes in its 2017 children’s shows <a href="https://www.theverge.com/2017/6/20/15834858/netflix-interactive-shows-puss-in-boots-buddy-thunderstruck">Puss in Books</a> and <a href="https://www.theverge.com/2017/7/28/16055642/netflix-interactive-buddy-thunderstruck-children-tv-programming">Buddy Thunderstruck: The Maybe Pile</a>. While well-received, they remain in the sphere of children’s shows, with the format presented as a way of gamifying TV, arguably “<a href="https://variety.com/2017/digital/news/netflix-puss-in-book-interactive-1202471301/">to make kids TV shows even more addictive</a>”.</p>
<figure>
<iframe width="440" height="260" src="https://www.youtube.com/embed/YtvkFv7igno?wmode=transparent&start=0" frameborder="0" allowfullscreen=""></iframe>
</figure>
<p>But this interactive format appeared even earlier. Back in 1967, the Czechoslovakia pavilion at the Montreal Expo exhibited a film called Kinoautomat. Created by <a href="https://en.wikipedia.org/wiki/Rad%C3%BAz_%C4%8Cin%C4%8Dera">Radúz Činčera</a>, the film was screened in a specially built cinema hall with a green and red button installed at each seat. At various points during the film, the reel stops, and a moderator appears on stage to ask the audience to choose between two scenes. The audience’s votes are then reflected in green or red lights around the screen, and the scene with more votes is played.</p>
<p>No matter what choices were made, the film always had the same ending (a burning building). Arising out of 1960s Communist Czechoslovokia, Kinoautomat thus not only made a statement about the validity of branching story path structures – but also a profound one on the validity of political choice and the nature of democracy.</p>
<p>More recently, the interactive web series, <a href="https://www.trylife.tv/">Try Life</a>, created in 2012, presents similar episodes on teenage life. Taking on problematic life issues such as abuse, drug use and violence, a choice could quickly end up in chaos or redemption. Like Kinoautomat, the format underscores how choices are not frivolous entertainment. A mistake could lead to terrible consequences: one cannot “try” life.</p>
<figure class="align-center ">
<img alt="" src="https://images.theconversation.com/files/254310/original/file-20190117-32831-408taz.PNG?ixlib=rb-1.1.0&q=45&auto=format&w=754&fit=clip" srcset="https://images.theconversation.com/files/254310/original/file-20190117-32831-408taz.PNG?ixlib=rb-1.1.0&q=45&auto=format&w=600&h=273&fit=crop&dpr=1 600w, https://images.theconversation.com/files/254310/original/file-20190117-32831-408taz.PNG?ixlib=rb-1.1.0&q=30&auto=format&w=600&h=273&fit=crop&dpr=2 1200w, https://images.theconversation.com/files/254310/original/file-20190117-32831-408taz.PNG?ixlib=rb-1.1.0&q=15&auto=format&w=600&h=273&fit=crop&dpr=3 1800w, https://images.theconversation.com/files/254310/original/file-20190117-32831-408taz.PNG?ixlib=rb-1.1.0&q=45&auto=format&w=754&h=343&fit=crop&dpr=1 754w, https://images.theconversation.com/files/254310/original/file-20190117-32831-408taz.PNG?ixlib=rb-1.1.0&q=30&auto=format&w=754&h=343&fit=crop&dpr=2 1508w, https://images.theconversation.com/files/254310/original/file-20190117-32831-408taz.PNG?ixlib=rb-1.1.0&q=15&auto=format&w=754&h=343&fit=crop&dpr=3 2262w" sizes="(min-width: 1466px) 754px, (max-width: 599px) 100vw, (min-width: 600px) 600px, 237px">
<figcaption>
<span class="caption">Time-bar in Bandersnatch: pick before the algorithm does it for you.</span>
<span class="attribution"><span class="source">Netflix</span></span>
</figcaption>
</figure>
<p>Even the time-bar in Bandersnatch – a rectangular block presented alongside each choice which shortens in length to signify the amount of time left to the viewer to choose (before Netflix’s algorithm does it for them) – has been seen before. In 2016, CtrlMovie AG released a smartphone app on which viewers can play an interactive movie called <a href="http://lateshift-movie.com/app/">Late Shift</a>. It displays exactly the same feature.</p>
<figure class="align-center ">
<img alt="" src="https://images.theconversation.com/files/254309/original/file-20190117-32837-nusesz.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=754&fit=clip" srcset="https://images.theconversation.com/files/254309/original/file-20190117-32837-nusesz.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=600&h=338&fit=crop&dpr=1 600w, https://images.theconversation.com/files/254309/original/file-20190117-32837-nusesz.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=600&h=338&fit=crop&dpr=2 1200w, https://images.theconversation.com/files/254309/original/file-20190117-32837-nusesz.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=600&h=338&fit=crop&dpr=3 1800w, https://images.theconversation.com/files/254309/original/file-20190117-32837-nusesz.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=754&h=424&fit=crop&dpr=1 754w, https://images.theconversation.com/files/254309/original/file-20190117-32837-nusesz.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=754&h=424&fit=crop&dpr=2 1508w, https://images.theconversation.com/files/254309/original/file-20190117-32837-nusesz.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=754&h=424&fit=crop&dpr=3 2262w" sizes="(min-width: 1466px) 754px, (max-width: 599px) 100vw, (min-width: 600px) 600px, 237px">
<figcaption>
<span class="caption">Time-bar in Late Shift: up to you which scene to pick.</span>
<span class="attribution"><span class="source">CtrlMovie AG</span></span>
</figcaption>
</figure>
<h2>All about algorithms</h2>
<p>So what is different about Bandersnatch? Its interactive storytelling format has been done before – and better. Instead, it’s the less glamorous issue of Netflix’s back-end programming that makes Bandersnatch an exceptional storytelling experience. What Bandersnatch <em>really</em> shows is how Netflix’s algorithms are able to deliver an unprecedentedly seamless experience of processing and presenting branching story paths for individual viewing experiences on a platform streaming to more than 100m subscribers worldwide.</p>
<p>When you think back to Kinoautomat in the bespoke cinema hall – where at each stop a projectionist in the backroom had to set up the chosen reel in a hurry to play the relevant choice – it’s clear how far we’ve come.</p>
<p>And this is where interactive storytelling will make its biggest breakthrough in the next decade – not by presenting interfaces of choices for viewers, but in how computational algorithms will be automated to such a degree of sophistication that they will be able to process and produce audiovisual media with which to tell our stories, in all the ways with which we as humans use stories to laugh, cry, think and make sense of our lives.</p>
<figure>
<iframe width="440" height="260" src="https://www.youtube.com/embed/gJEzuYynaiw?wmode=transparent&start=0" frameborder="0" allowfullscreen=""></iframe>
</figure>
<p>There is already evidence of algorithmic narrative power to demonstrate this potential. In 2016, IBM produced the first film trailer <a href="https://www.wired.co.uk/article/ibm-watson-ai-film-trailer">created entirely by artificial intelligence</a>. To make the trailer for 20th Century Fox thriller Morgan a database of thriller trailers were fed into the IBM Watson computer. Through pattern-finding and other functions, the algorithm then selected music and scenes from the film to piece together <a href="https://www.youtube.com/watch?v=gJEzuYynaiw">a credible trailer</a>. </p>
<p>Thanks to the accuracy of today’s computational language generators, it’s hard to work out whether some writing is <a href="http://botpoet.com/">created by humans or computers</a>. </p>
<p>As consumers and citizens, we need to understand how computational automation is able to process language, emotion, morality, personality and other fundamental human traits. We need to do this quickly, because increasingly it will be these algorithms that are telling our stories in future, while we become increasingly passive partners. And we need to work out whether this is what we want.</p><img src="https://counter.theconversation.com/content/109830/count.gif" alt="The Conversation" width="1" height="1" />
<p class="fine-print"><em><span>Jenna Ng does not work for, consult, own shares in or receive funding from any company or organisation that would benefit from this article, and has disclosed no relevant affiliations beyond their academic appointment.</span></em></p>With the latest series of Black Mirror, Netflix is driving television to new levels of interactivity.Jenna Ng, Anniversary Research Lecturer in Film and Interactive Media, University of YorkLicensed as Creative Commons – attribution, no derivatives.tag:theconversation.com,2011:article/949662018-04-24T19:06:56Z2018-04-24T19:06:56ZThe hypodermic effect: How propaganda manipulates our emotions<figure><img src="https://images.theconversation.com/files/215824/original/file-20180422-75104-1n8b5ob.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=496&fit=clip" /><figcaption><span class="caption">Facebook CEO Mark Zuckerberg departs after testifying on Capitol Hill in Washington, D.C., in April 2018 about the use of Facebook data to target American voters in the 2016 presidential election and data privacy. </span> <span class="attribution"><span class="source">(AP Photo/Andrew Harnik)</span></span></figcaption></figure><p>The scandal surrounding the improper use of data by <a href="https://www.reuters.com/article/us-facebook-cambridge-analytica/cambridge-analytica-ceo-claims-influence-on-u-s-election-facebook-questioned-idUSKBN1GW1SG">Cambridge Analytica and Facebook in the 2016 U.S. election</a> is reminiscent of the old debates about propaganda and its ability to “violate the minds of the masses,” according to <a href="https://ipfs.io/ipfs/QmXoypizjW3WknFiJnKLwHCnL72vedxjQkDDP1mXWo6uco/wiki/Sergei_Chakhotin.html">Sergei Tchakhotin,</a> an expert in the study of Nazi propaganda.</p>
<p>The Russian sociologist said that the masses were subjected to a sophisticated machinery of manipulation that could, through the strategic use of radio, film and well-orchestrated performances, touch on and influence the basic instincts of Germans.</p>
<p>Decades later, we’re once again back discussing the manipulation of emotions, this time via social media platforms. </p>
<figure class="align-right ">
<img alt="" src="https://images.theconversation.com/files/215973/original/file-20180423-94115-qux4ip.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=237&fit=clip" srcset="https://images.theconversation.com/files/215973/original/file-20180423-94115-qux4ip.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=600&h=775&fit=crop&dpr=1 600w, https://images.theconversation.com/files/215973/original/file-20180423-94115-qux4ip.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=600&h=775&fit=crop&dpr=2 1200w, https://images.theconversation.com/files/215973/original/file-20180423-94115-qux4ip.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=600&h=775&fit=crop&dpr=3 1800w, https://images.theconversation.com/files/215973/original/file-20180423-94115-qux4ip.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=754&h=974&fit=crop&dpr=1 754w, https://images.theconversation.com/files/215973/original/file-20180423-94115-qux4ip.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=754&h=974&fit=crop&dpr=2 1508w, https://images.theconversation.com/files/215973/original/file-20180423-94115-qux4ip.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=754&h=974&fit=crop&dpr=3 2262w" sizes="(min-width: 1466px) 754px, (max-width: 599px) 100vw, (min-width: 600px) 600px, 237px">
<figcaption>
<span class="caption">Nazi propaganda minister Dr. Joseph Goebbels is seen in this October 1938 photo as he speaks to members of the Nazi party.</span>
<span class="attribution"><span class="source">(AP Photo)</span></span>
</figcaption>
</figure>
<p>Of course, the communication ecosystem is very different from what existed for Joseph Goebbels, Hitler’s propaganda minister. But the underlying principles for manipulating the masses do not seem to have changed much. </p>
<p>Reports indicate that <a href="https://www.newyorker.com/news/news-desk/cambridge-analytica-and-the-perils-of-psychographics">Cambridge Analytica</a> developed a methodology that allowed them to establish psychographic profiles of Facebook users, and thus push emotional buttons that could influence their political preferences and voting behaviour. </p>
<p>To some degree, this represents the return of what’s known as the <a href="https://www.utwente.nl/en/bms/communication-theories/sorted-by-cluster/Mass%20Media/Hypodermic_Needle_Theory/">hypodermic effect</a> in which the audience falls “victim” to powerful media that have the ability to manipulate our emotions and shape our understanding of the world.</p>
<hr>
<p>
<em>
<strong>
Read more:
<a href="https://theconversation.com/why-we-should-all-cut-the-facebook-cord-or-should-we-93929">Why we should all cut the Facebook cord. Or should we?</a>
</strong>
</em>
</p>
<hr>
<p>Research, however, indicates that how we respond to <a href="https://www.britannica.com/topic/two-step-flow-model-of-communication#ref1199152">media does not adhere to what’s known as a stimulus-response causality</a>. There are other factors that intervene in the way people use, perceive and process what they consume in the media. They are known as “mediations” that, according to the Spanish-Colombian professor Jesús Martín Barbero, are the <a href="http://journals.sagepub.com/doi/abs/10.1177/1742766506069579">different ways people interpret the messages conveyed by the media</a>.</p>
<h2>Using our data to influence us</h2>
<p>But today, governments, corporations and political parties have the unprecedented ability to process a litany of data and then, through sophisticated algorithms, broadcast messages and images to influence an increasingly segmented audience. </p>
<p>One must ask, then, what role will Martín Barbero’s mediations — our cultural references, values, family, friends and other reference groups that influence our reading of the mediated messages — play in how we consume information and entertainment on social networks? </p>
<p>Are we condemned to live the “dystopian realism” presented by the British TV series <em>Black Mirror</em> in which <a href="https://www.theguardian.com/technology/2011/dec/01/charlie-brooker-dark-side-gadget-addiction-black-mirror">digital media penetrate the intimacy of a human being too clumsy to resist the temptation of being manipulated</a>, according to the show’s creator Charlie Brooker?</p>
<hr>
<p>
<em>
<strong>
Read more:
<a href="https://theconversation.com/god-is-an-algorithm-why-were-closer-to-a-black-mirror-style-reality-than-we-think-90669">God is an algorithm: why we're closer to a Black Mirror-style reality than we think</a>
</strong>
</em>
</p>
<hr>
<p>The debate about the influence of Facebook and unscrupulous companies like Cambridge Analytica reveals the importance of emotions not only in our private lives but also in our so-called “public lives” as citizens. The problem arises in terms not only of “emotional manipulation” but of the role emotions play in how we relate and understand the world around us.</p>
<p>As the <a href="https://elpais.com/elpais/2018/03/27/ciencia/1522150428_248366.html">neuroscientist Antonio Damasio recently said</a>: </p>
<blockquote>
<p>“Culture works by a system of selection similar to that of genetic selection, except that what is being selected is an instrument that we put into practice. Feelings are an agent in cultural selection. I think that the beauty of the idea is in seeing feelings as motivators, as a surveillance system, and as negotiators.”</p>
</blockquote>
<p>If feelings are an integral part of this “cultural selection,” are we facing a shift in this sociocultural evolutionary process due to the “algorithmization” of emotions? </p>
<p>Is historian Yuval Noah Harari right when he says that “technological religion” — <a href="https://www.ft.com/content/50bb4830-6a4c-11e6-ae5b-a7cc5dd5a28c">he calls it “dataism”</a> — is transforming us in such a way that it will make the homo sapiens irrelevant and put the human being on the periphery in a world dominated by algorithms?</p>
<h2>More isolation ahead?</h2>
<p>These are complex questions that are difficult to answer. </p>
<p>In any case, it seems that our intellectual or even emotional laziness is transforming us into puppets of our emotions. Evidence is emerging that digital media is changing the configuration of our nervous system and our forms of socialization. </p>
<p>Sherry Turkle, a professor at MIT, observes in her book <a href="https://www.theguardian.com/books/2011/jan/30/alone-together-sherry-turkle-review"><em>Alone together: Why we expect more from technology and less from each other</em></a> that there are already signs of dissatisfaction among young people who are obsessed with their image on social media while losing the ability of introspection; mothers who feel that communication with their children via text messages is more frequent but less substantive; and Facebook users who think that the banalities they share with their “virtual friends” devalue the true intimacy between friends. </p>
<p>If virtual relations replace face-to-face contact, we may see more isolation, individualism and less social cohesion, which does not bode well for the survival of democracy. </p>
<p>It’s also likely that the expansion of social media does not make us more rational. Although we have access to more information and participate in more public debates about issues that affect us as individuals and as a society, that doesn’t mean we’re doing so more rationally or based on arguments that are scientifically factual.</p>
<p>The rise of religious fundamentalism, nationalism, of beliefs in all kinds of sects and New Age fashions are symptoms of a “return of sorcerers” or magical thinking in our digital society. </p>
<p>We deploy our egos on social media, sometimes with a compulsive need for recognition. This knowledge of our self, quantified in big data and transformed into affective algorithms, is exploited by corporations and political parties to give us, as <a href="https://www.phrases.org.uk/meanings/fifteen-minutes-of-fame.html">Andy Warhol said, our 15 minutes of fame</a>. </p>
<p>The sorcerers of propaganda are back — this time with more powerful means that their predecessors.</p><img src="https://counter.theconversation.com/content/94966/count.gif" alt="The Conversation" width="1" height="1" />
<p class="fine-print"><em><span>Isaac Nahon-Serfaty has received funding from the Social Sciences and Humanities Research Council of Canada and the Canadian Institutes of Health Research.</span></em></p>Knowledge of our selves, quantified in big data and transformed into affective algorithms, is exploited by corporations and political parties to give us our 15 minutes of fame.Isaac Nahon-Serfaty, Associate Professor, L’Université d’Ottawa/University of OttawaLicensed as Creative Commons – attribution, no derivatives.tag:theconversation.com,2011:article/906592018-02-28T13:50:53Z2018-02-28T13:50:53ZHow Black Mirror combines a disturbing future with a familiar past<figure><img src="https://images.theconversation.com/files/208050/original/file-20180227-36671-1o6a5x5.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=496&fit=clip" /><figcaption><span class="caption">
</span> <span class="attribution"><span class="source">Netflix</span></span></figcaption></figure><p>The Netflix series <a href="https://www.netflix.com/gb/title/70264888">Black Mirror</a> is a supreme example of contemporary television fantasy. It is hard to think of any other modern show that is so constantly unpredictable, aesthetically accomplished, stylistically eclectic or downright disturbing. </p>
<p>Over 19 self-contained episodes split into four series, Black Mirror seamlessly blurs genre boundaries – science fiction, horror, thriller and satire all meld into one another. </p>
<p>But despite the eclectic sense of genre, Black Mirror is united by the theme of digital technology. The name of the programme itself refers to a turned off phone, television or any of the other screens that dominate our lives – the haunting black screen like a mirror. It seeks to identify the radical changes brought about by digital technology and push them to their logical conclusion. </p>
<p>Just as the pioneering science fiction of the 19th century incorporated the scientific discoveries and technology of the industrial revolution, so Black Mirror pushes our contemporary experience of the digital revolution into possible projections of the future. </p>
<figure>
<iframe width="440" height="260" src="https://www.youtube.com/embed/5ELQ6u_5YYM?wmode=transparent&start=0" frameborder="0" allowfullscreen=""></iframe>
</figure>
<p>Some episodes of Black Mirror can even be seen as adaptations of much older literature. One striking example is <a href="http://www.imdb.com/title/tt2290780/">Be Right Back</a> from series two, a clever reworking of <a href="https://theconversation.com/frankenstein-at-200-and-why-mary-shelley-was-far-more-than-the-sum-of-her-monsters-parts-90206">Mary Shelley’s Frankenstein</a> from 1818. </p>
<p>In the Black Mirror version, after a man is killed in a car accident, his distraught widow purchases a biosynthetic model of her husband. It downloads all the extant digital documentation of her late husband and comes alive as a seemingly faultless imitation of the dead man. </p>
<p>Tragedy is inevitable of course. Despite his superficial perfection (it is even, in some respects, an improvement on the original), the resurrected figure – like Frankenstein’s creature – can never be human. And once she has passed through the euphoria and solace she finds in the biosimulation, the widow gradually comes to regard her reconstructed partner with horror. </p>
<p>Be Right Back is an effective re-imagining of Frankenstein for the 21st century. It updates Shelley’s ethical questions to assert that the digital presence which surrounds us is no more the vital essence of humanity than Frankenstein’s assemblage of body parts. But while Frankenstein explores the tragic relationship between the creation and his creator, Be Right Back focuses on the creation and the consumer.</p>
<p>That consumer is also centre stage in the satirical comedy <a href="http://www.imdb.com/title/tt5497778/">Nosedive</a>, from series three. Set in a not so distant future where social media “likes” determine a person’s job, lifestyle and prospects, its story could have been framed within any number of genres. Here it is presented as a sardonic farce in which a young woman has a journey from hell travelling to a wedding where she is due to be maid of honour. </p>
<p>Far from comedic is <a href="http://www.imdb.com/title/tt5710984/">Metalhead</a> (series four), a tale of ultra-surveillance and materialism. It is told as a super-stylish monochrome horror story in which security systems have created anti-theft machines that have become so vigilant they have killed off nearly all living creatures. </p>
<p>The supreme irony in this episode is that the dystopia has not destroyed the very materialism that created the surveillance technology in the first place. When it is revealed what the group of human survivors were risking their lives for, we realise that although it was emotive and humane, it seemed an absurd risk to take. </p>
<figure class="align-center ">
<img alt="" src="https://images.theconversation.com/files/208043/original/file-20180227-36686-1uj3w0n.jpg?ixlib=rb-1.1.0&rect=34%2C21%2C2841%2C1427&q=45&auto=format&w=754&fit=clip" srcset="https://images.theconversation.com/files/208043/original/file-20180227-36686-1uj3w0n.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=600&h=400&fit=crop&dpr=1 600w, https://images.theconversation.com/files/208043/original/file-20180227-36686-1uj3w0n.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=600&h=400&fit=crop&dpr=2 1200w, https://images.theconversation.com/files/208043/original/file-20180227-36686-1uj3w0n.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=600&h=400&fit=crop&dpr=3 1800w, https://images.theconversation.com/files/208043/original/file-20180227-36686-1uj3w0n.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=754&h=503&fit=crop&dpr=1 754w, https://images.theconversation.com/files/208043/original/file-20180227-36686-1uj3w0n.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=754&h=503&fit=crop&dpr=2 1508w, https://images.theconversation.com/files/208043/original/file-20180227-36686-1uj3w0n.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=754&h=503&fit=crop&dpr=3 2262w" sizes="(min-width: 1466px) 754px, (max-width: 599px) 100vw, (min-width: 600px) 600px, 237px">
<figcaption>
<span class="caption">Brooker: the man behind the mirror.</span>
<span class="attribution"><span class="source">Netflix</span></span>
</figcaption>
</figure>
<h2>Dark reflections</h2>
<p>Black Mirror is the creation of <a href="http://www.imdb.com/name/nm0111765/bio?ref_=nm_ov_bio_sm">Charlie Brooker</a>, a figure formerly best known for his satirical journalism and biting assessments of popular culture. Appearing on BBC’s Desert Island Discs radio programme, <a href="http://www.bbc.co.uk/programmes/b09kx840">he recently recounted</a> how he spent years screaming at the television: “I could do better than that!” With Black Mirror, he probably has. </p>
<p>As well as drawing on the foundations of science fiction, Black Mirror belongs to the rich tradition of dark fantasy on television. Even the monochrome opening sequence of a buffering signal and cracking screen pays homage to the opening credits of <a href="http://www.imdb.com/title/tt0056777/">The Outer Limits</a> – the 1960s TV series which informed viewers they had lost control of their televisions. </p>
<figure>
<iframe width="440" height="260" src="https://www.youtube.com/embed/ORbseYAkzRM?wmode=transparent&start=0" frameborder="0" allowfullscreen=""></iframe>
</figure>
<p>The Outer Limits and Black Mirror both draw emphatic attention to the very technology being used by viewers. Half a century ago, The Outer Limits (and the even more successful <a href="http://www.imdb.com/title/tt0052520/?ref_=nv_sr_1">The Twilight Zone</a>) used television to present metaphors of social anxieties, including communism, the loss of social values and technophobia. These were quintessential examples of television fantasy. Where better to disquiet and unnerve us than in the safety and comfort of our own homes? </p>
<p>Black Mirror is a direct descendant of these shows. It examines the increasingly complex technological systems of our domestic environments and ways of living in the 21st century – making both seem dangerous and fragile.</p><img src="https://counter.theconversation.com/content/90659/count.gif" alt="The Conversation" width="1" height="1" />
<p class="fine-print"><em><span>Richard Hand does not work for, consult, own shares in or receive funding from any company or organisation that would benefit from this article, and has disclosed no relevant affiliations beyond their academic appointment.</span></em></p>Netflix hit, Black Mirror, follows in the footsteps of other forward-thinking sci-fi storytellers.Richard Hand, Professor of Media Practice, University of East AngliaLicensed as Creative Commons – attribution, no derivatives.tag:theconversation.com,2011:article/906692018-01-31T13:36:36Z2018-01-31T13:36:36ZGod is an algorithm: why we’re closer to a Black Mirror-style reality than we think<figure><img src="https://images.theconversation.com/files/204230/original/file-20180131-131721-1baq0cc.png?ixlib=rb-1.1.0&q=45&auto=format&w=496&fit=clip" /><figcaption><span class="caption">Black Mirror</span> </figcaption></figure><p><em>This article contains spoilers about Black Mirror, season four</em></p>
<p>Do we have free will, or are we controlled by a higher power? The capacity to act and determine one’s own actions in an increasingly technologised world is the most prominent theme in the latest season of <a href="https://www.dailydot.com/upstream/what-is-netflix/">Netflix’s</a> <a href="https://www.theguardian.com/tv-and-radio/2015/feb/06/black-mirror-charlie-brooker-box-set-review">Black Mirror</a>. And the question writer <a href="http://www.imdb.com/name/nm0111765/bio?ref_=nm_ov_bio_sm">Charlie Brooker</a> addresses in his bleak sketches is as old as human consciousness itself.</p>
<p>Before the <a href="http://www.britishmuseum.org/research/publications/online_research_catalogues/paper_money/paper_money_of_england__wales/the_industrial_revolution.aspx">Industrial Revolution</a> and the first sci-fi narrative (Mary Shelley’s <a href="https://www.theguardian.com/books/2018/jan/13/frankenstein-at-200-why-hasnt-mary-shelley-been-given-the-respect-she-deserves-">Frankenstein</a>, 200 years old this year) warning us of the dangers of replacing an inscrutable ancient god with a scientific one, people tried to determine two opposing but closely related things: how much agency they had, and whether they could rely on miraculous help from above in a time of difficulty. For free will is both a burden and a blessing. While we are imperfect, our vision limited by our perception, surely God is omniscient, omnipotent and wise? When we’re in trouble, he can save us and redeem our mistakes. </p>
<p>Isn’t technology a better god than the previous god? In the past, people asked deities about weather patterns, love, luck, and everything else. Their predictions worked, at best, 50% per cent of the time. Now we have weather forecasts delivered to our mobiles, predictive dating apps and GPS trackers. Don’t we all want to live in a world in which our fallible agency is replaced by technological perfection?</p>
<figure>
<iframe width="440" height="260" src="https://www.youtube.com/embed/U2YPxSDIoPE?wmode=transparent&start=0" frameborder="0" allowfullscreen=""></iframe>
<figcaption><span class="caption">YouTube.</span></figcaption>
</figure>
<p>Brooker’s answer to this question is a resounding “no”. Technology is certainly a more efficient god since it has turned magic into reality, but the human issue of free will is still a big part of our relationship with it. Seeing how clever, precise, omniscient and infallible this new god is, we have decided to entrust it with a range of mundane tasks we previously used to perform ourselves: counting, translating, finding our way, even expressing emotions and, of course, shopping.</p>
<h2>A mirror to reality?</h2>
<p>This is a tendency Brooker particularly despises. In his view, human laziness is what is going to destroy civilisation. Brooker predicts a world in which the scope of action for free will – our error-prone but nevertheless important decision-making capacity – becomes so narrow that we forget how it feels to be human, how to feel pain and to make mistakes. Thanks to smart phones, tablets, <a href="https://thewirecutter.com/reviews/what-is-alexa-what-is-the-amazon-echo-and-should-you-get-one/">Alexa</a> and Google, we have cognitively unloaded anything requiring an effort or possessing a margin for error to <a href="http://www.bbc.co.uk/timelines/zq376fr">artificial intelligence</a> (AI). All these functions will now be taken care of by the god of technology, by the All-Seeing Algorithm.</p>
<p>Even the more optimistic episodes of the latest (fourth) season – Hang the DJ and Black Museum – show that the human desire for an all-controlling, all-knowing supreme being does, indeed, result in exactly this kind of supreme being, but not in a good way.</p>
<p>In Hang the DJ, we are shown a world in which finding a mate no longer involves going through a series of disappointments and bouts of happiness. An app finds a person’s perfect match while their “copies”, trapped within what is called “the system”, make mistakes and suffer broken hearts instead of their “originals” who are waiting for the result in real life.</p>
<p>Although the episode’s finale is unexpectedly uplifting and positive, its overall message is not: as people, we have gone too far in shielding ourselves from any errors in the decision-making process. We wanted more perfection and less agency, and that’s exactly what we got.</p>
<figure class="align-center ">
<img alt="" src="https://images.theconversation.com/files/204247/original/file-20180131-131727-x5my5h.png?ixlib=rb-1.1.0&q=45&auto=format&w=754&fit=clip" srcset="https://images.theconversation.com/files/204247/original/file-20180131-131727-x5my5h.png?ixlib=rb-1.1.0&q=45&auto=format&w=600&h=365&fit=crop&dpr=1 600w, https://images.theconversation.com/files/204247/original/file-20180131-131727-x5my5h.png?ixlib=rb-1.1.0&q=30&auto=format&w=600&h=365&fit=crop&dpr=2 1200w, https://images.theconversation.com/files/204247/original/file-20180131-131727-x5my5h.png?ixlib=rb-1.1.0&q=15&auto=format&w=600&h=365&fit=crop&dpr=3 1800w, https://images.theconversation.com/files/204247/original/file-20180131-131727-x5my5h.png?ixlib=rb-1.1.0&q=45&auto=format&w=754&h=459&fit=crop&dpr=1 754w, https://images.theconversation.com/files/204247/original/file-20180131-131727-x5my5h.png?ixlib=rb-1.1.0&q=30&auto=format&w=754&h=459&fit=crop&dpr=2 1508w, https://images.theconversation.com/files/204247/original/file-20180131-131727-x5my5h.png?ixlib=rb-1.1.0&q=15&auto=format&w=754&h=459&fit=crop&dpr=3 2262w" sizes="(min-width: 1466px) 754px, (max-width: 599px) 100vw, (min-width: 600px) 600px, 237px">
<figcaption>
<span class="caption">In an episode from the new season of Black Mirror, a woman uses a surveillance implant to monitor her daughter’s safety with disastrous consequences as the girl gets older.</span>
<span class="attribution"><span class="source">Netflix</span></span>
</figcaption>
</figure>
<p>This perfection, however, comes at a price. In Arkangel, a mother implants into her baby daughter a tracking device which monitors her well-being and detects her location. The device also allows the mother to see the world through her daughter’s eyes and to blur out any disturbing information. However, the daughter fights for her right to make mistakes and to handle the unpleasantness of the world. Keen to break up the unhealthy attachment aided by technology, the girl ends up taking drugs and having sex. Instead of the flawless child, the mother is faced with a rebellious teenager who ends up beating up and leaving her over-protective parent.</p>
<h2>When good technology goes bad</h2>
<p>Instead of being helpful and protective, technology becomes terrifying in Metalhead – a stark black-and-white vignette reminiscent of The Terminator – in which a sole female survivor is pursued by a robotic dog-like creature after a failed warehouse raid. The dog is autonomous, relentless and problem-solving. It does not make mistakes. Although the protagonist manages to outsmart the canine terminator on a number of occasions, in the end the technology is so powerful and ubiquitous that killing herself is her only escape.</p>
<p>Those hoping to survive the onslaught of technological precision must look for rare flaws in it. This is what happens in USS Callister – probably the best episode in the series. Copies of real people trapped inside a private version of a space video game attempt to escape from it through a wormhole which temporarily appears, only to find themselves in the commercial version of the same game. Although they have more agency, they are still not entirely free.</p>
<p>And again, in Crocodile, the technology does not fail, but the human being does, try as she might. The recently invented memory-retrieving device prevents Mia from concealing the murders she has committed. Interestingly enough, Brooker makes us sympathise with the murderer as we witness her struggle to evade the relentless power of technology. Her agency is thwarted by the device, leaving her with no chance of escape. </p>
<p>In all episodes, Brooker shows the inevitable end of human agency as daily routines are taken over by artificial intelligence. Technology leaves only a small margin for human error. This is an excellent god. It has realised so many of our dreams, but is it the god we wanted?</p>
<p>Perhaps we should trust ourselves more and rely less on unfailing and dependable technology. Surrendering to it means losing that vital element that makes us human: the ability to make mistakes and to grow by learning from them.</p><img src="https://counter.theconversation.com/content/90669/count.gif" alt="The Conversation" width="1" height="1" />
<p class="fine-print"><em><span>Helena Bassil-Morozow does not work for, consult, own shares in or receive funding from any company or organisation that would benefit from this article, and has disclosed no relevant affiliations beyond their academic appointment.</span></em></p>By surrendering to technology are humans sleepwalking into a future where free will is less and less of an option?Helena Bassil-Morozow, Lecturer in Media and Journalism, Glasgow Caledonian UniversityLicensed as Creative Commons – attribution, no derivatives.