tag:theconversation.com,2011:/us/topics/google-nest-92617/articlesGoogle Nest – The Conversation2022-07-14T13:43:45Ztag:theconversation.com,2011:article/1868742022-07-14T13:43:45Z2022-07-14T13:43:45Z‘Hey Siri’: Virtual assistants are listening to children and then using the data<figure><img src="https://images.theconversation.com/files/473758/original/file-20220713-26-qoaw40.jpg?ixlib=rb-1.1.0&rect=0%2C0%2C5499%2C3661&q=45&auto=format&w=496&fit=clip" /><figcaption><span class="caption">Virtual assistants are becoming a more common household fixture, and many children are growing up and interacting with them.</span> <span class="attribution"><span class="source">(Shutterstock)</span></span></figcaption></figure><p>In many busy households around the world, it’s not uncommon for children to shout out directives to Apple’s Siri or Amazon’s Alexa. They may make a game out of asking the voice-activated personal assistant (VAPA) what time it is, or requesting a popular song. While this may seem like a mundane part of domestic life, there is much more going on.</p>
<p>The VAPAs are continuously listening, recording and processing acoustic happenings in a process that has been dubbed “<a href="https://doi.org/10.24908/ss.v18i3.13426">eavesmining</a>,” a portmanteau of eavesdropping and datamining. This raises significant concerns pertaining to issues of <a href="https://doi.org/10.24908/ss.v17i1/2.12936">privacy and surveillance</a>, as well as <a href="https://doi.org/10.7551/mitpress/10993.003.0013">discrimination</a>, as the sonic traces of peoples’ lives become datafied and scrutinized by algorithms.</p>
<p>These concerns intensify as we apply them to children. Their data is accumulated over lifetimes in ways that go well beyond what was ever collected on their parents with far-reaching consequences that we haven’t even begun to understand.</p>
<hr>
<p>
<em>
<strong>
Read more:
<a href="https://theconversation.com/we-street-proof-our-kids-why-arent-we-data-proofing-them-123415">We street-proof our kids. Why aren't we data-proofing them?</a>
</strong>
</em>
</p>
<hr>
<h2>Always listening</h2>
<p>The adoption of VAPAs is proceeding at a staggering pace as it extends to include mobile phones, smart speakers and the ever-increasing number products that are connected to the internet. These include <a href="https://www.washingtonpost.com/news/the-switch/wp/2015/03/11/privacy-advocates-try-to-keep-creepy-eavesdropping-hello-barbie-from-hitting-shelves/">children’s digital toys</a>, <a href="https://www.amazon.com/gp/help/customer/display.html?nodeId=G48H6GVHV769T9Z5">home security systems that listen for break-ins</a> and <a href="https://www.bloomberg.com/opinion/articles/2021-10-15/amazon-s-ring-doorbell-camera-raises-lots-of-privacy-alarms">smart doorbells that can pickup sidewalk conversations</a>.</p>
<p>There are pressing issues that derive from the collection, storage and analysis of sonic data as they pertain to parents, youth and children. Alarms have been raised in the past — in 2014, privacy advocates raised concerns on how much the <a href="https://www.slashgear.com/how-private-is-amazon-echo-07354486">Amazon Echo</a> was listening to, what data was being collected and how the data would be used by Amazon’s recommendation engines. </p>
<p>And yet, despite these concerns, VAPAs and other eavesmining systems have spread exponentially. Recent market research predicts that by 2024, <a href="https://www.juniperresearch.com/press/number-of-voice-assistant-devices-in-use">the number of voice-activated devices will explode to over 8.4 billion</a>. </p>
<figure class="align-center zoomable">
<a href="https://images.theconversation.com/files/473759/original/file-20220713-18-prkevj.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=1000&fit=clip"><img alt="a young girl sits with her head in her hands as two adults yell ateach other in the background" src="https://images.theconversation.com/files/473759/original/file-20220713-18-prkevj.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=754&fit=clip" srcset="https://images.theconversation.com/files/473759/original/file-20220713-18-prkevj.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=600&h=399&fit=crop&dpr=1 600w, https://images.theconversation.com/files/473759/original/file-20220713-18-prkevj.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=600&h=399&fit=crop&dpr=2 1200w, https://images.theconversation.com/files/473759/original/file-20220713-18-prkevj.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=600&h=399&fit=crop&dpr=3 1800w, https://images.theconversation.com/files/473759/original/file-20220713-18-prkevj.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=754&h=501&fit=crop&dpr=1 754w, https://images.theconversation.com/files/473759/original/file-20220713-18-prkevj.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=754&h=501&fit=crop&dpr=2 1508w, https://images.theconversation.com/files/473759/original/file-20220713-18-prkevj.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=754&h=501&fit=crop&dpr=3 2262w" sizes="(min-width: 1466px) 754px, (max-width: 599px) 100vw, (min-width: 600px) 600px, 237px"></a>
<figcaption>
<span class="caption">Even when virtual assistants aren’t actively on, they’re listening to and recording their environments.</span>
<span class="attribution"><span class="source">(Shutterstock)</span></span>
</figcaption>
</figure>
<h2>Recording more than just speech</h2>
<p><a href="https://yalebooks.yale.edu/book/9780300248036/the-voice-catchers/">There is more being gathered than just uttered statements</a>, as VAPAs and other eavesmining systems overhear personal features of voices that involuntarily reveal <a href="https://doi.org/10.1007/978-3-030-42504-3_16">biometric and behavioural attributes</a> such as age, gender, health, intoxication and personality. </p>
<p>Information about acoustic environments (like a noisy apartment) or particular sonic events (like breaking glass) can also be gleaned through “<a href="https://direct.mit.edu/books/book/3887/Auditory-Scene-AnalysisThe-Perceptual-Organization">auditory scene analysis</a>” to make judgments about what is happening in that environment. </p>
<p>Eavesmining systems already have a recent track record for <a href="https://www.radicalhistoryreview.org/abusablepast/surveying-the-suburbs-how-amazon-ring-and-a-racialized-fear-of-crime-is-ushering-in-a-new-period-of-mass-surveillance/">collaborating with law enforcement agencies</a> and <a href="https://www.cbc.ca/radio/day6/episode-417-alexa-as-murder-witness-k-tel-s-legacy-brexit-and-gibraltar-havana-s-mystery-hater-and-more-1.4916536/alexa-who-did-it-what-happens-when-a-judge-in-a-murder-trial-wants-data-from-a-smart-home-speaker-1.4916556">being subpoenaed for data in criminal investigations</a>. This raises concerns of other forms of <a href="https://doi.org/10.1215/01636545-2006-95-70">surveillance creep</a> and profiling of children and families. </p>
<p>For example, smart speaker data may be used to create profiles such as “noisy households,” “disciplinary parenting styles” or “troubled youth.” This could, in the future, be used by governments to profile those reliant on social assistance or families in crisis with potentially dire consequences. </p>
<p>There are also new eavesmining systems presented as a solution to keep children safe called “<a href="https://www.louroe.com/product/aggression-detector/">aggression detectors</a>.” <a href="https://www.soundintel.com/products/overview/aggression/">These technologies</a> consist of microphone systems loaded with machine learning software, dubiously claiming that they can help anticipate incidents of violence by listening for signs of raising volume and emotions in voices, and for other sounds such as glass breaking.</p>
<h2>Monitoring schools</h2>
<p>Aggression detectors are advertised in <a href="https://features.propublica.org/aggression-detector/the-unproven-invasive-surveillance-technology-schools-are-using-to-monitor-students/">school safety magazines and at law enforcement conventions</a>. They have been deployed in public spaces, hospitals and high schools under the guise of being able to pre-empt and detect mass shootings and other cases of lethal violence. </p>
<p>But there are serious issues around the efficacy and reliability of these systems. One brand of detector <a href="https://features.propublica.org/aggression-detector/the-unproven-invasive-surveillance-technology-schools-are-using-to-monitor-students/">repeatedly misinterpreted vocal cues of kids including coughing, screaming and cheering as indicators of aggression</a>. This begs the question of who is being protected and who will be made less safe by its design. </p>
<p>Some children and youth will be disproportionately harmed by this form of securitized listening, and the interests of all families will not be uniformly protected or served. A recurrent critique of voice-activated technology is that it <a href="https://liquidarchitecture.org.au/events/thao-phan-listening-to-misrecognition">reproduces cultural and racial biases by enforcing vocal norms and misrecognizing culturally diverse forms of speech in relation to language, accent, dialect and slang</a>. </p>
<figure>
<iframe width="440" height="260" src="https://www.youtube.com/embed/aS2Fp3W8l6A?wmode=transparent&start=0" frameborder="0" allowfullscreen=""></iframe>
<figcaption><span class="caption">Halcyon Lawrence, a technical communication scholar, expresses grave concerns about the potential deadly consequences that the use of voice technology alongside emotion recognition software will have on Black and brown people.</span></figcaption>
</figure>
<p>We can anticipate that the speech and voices of racialized children and youth will be disproportionately misinterpreted as aggressive sounding. This troubling prediction should come as no surprise as it follows the deeply entrenched colonial and white supremacist histories that consistently police a “<a href="https://nyupress.org/9781479889341/the-sonic-color-line/">sonic color line</a>.”</p>
<h2>Sound policy</h2>
<p>Eavesmining is a rich site of information and surveillance as children and families’ sonic activities have become valuable sources of data to be collected, monitored, stored, analysed and sold without the subject’s knowledge to thousands of third parties. These companies are profit-driven, with few ethical obligations to children and their data. </p>
<p>With no legal requirement to erase this data, the data accumulates over children’s lifetimes, potentially lasting forever. It is unknown how long and how far-reaching these digital traces will follow children as they age, how widespread this data will be shared or how much this data will be cross-referenced with other data. These questions have serious implications on children’s lives both presently and as they age.</p>
<p>There are a myriad threats posed by eavesmining in terms of privacy, surveillance and discrimination. Individualized recommendations, such as informational privacy education and digital literacy training, will be ineffective in addressing these problems and place too great a responsibility on families to develop the necessary literacies to counter eavesmining in public and private spaces.</p>
<p>We need to consider the advancement of a <a href="https://openvoicenetwork.org/">collective framework</a> that combats the unique risks and realities of eavesmining. Perhaps the development of a Fair Listening Practice Principles — an auditory spin on the “<a href="https://iapp.org/resources/article/fair-information-practices/">Fair Information Practice Principles</a>” — would help evaluate the platforms and processes that impact the sonic lives of children and families.</p><img src="https://counter.theconversation.com/content/186874/count.gif" alt="The Conversation" width="1" height="1" />
<p class="fine-print"><em><span>Stephen J. Neville receives funding from SSHRC.</span></em></p><p class="fine-print"><em><span>Natalie Coulter receives funding from SSHRC. </span></em></p>Children’s voices and actions are recorded by virtual assistants, but what is being done with all the collected information?Stephen J. Neville, PhD Student of Communication & Culture, York University, CanadaNatalie Coulter, Associate Professor of Communication Studies, and Director of the Institute for Research on Digital Literacies, York University, CanadaLicensed as Creative Commons – attribution, no derivatives.tag:theconversation.com,2011:article/1589212021-04-28T12:18:37Z2021-04-28T12:18:37ZShhhh, they’re listening – inside the coming voice-profiling revolution<figure><img src="https://images.theconversation.com/files/397406/original/file-20210427-13-sar48x.jpg?ixlib=rb-1.1.0&rect=5%2C2%2C1719%2C1127&q=45&auto=format&w=496&fit=clip" /><figcaption><span class="caption">Companies could soon tailor what they try to sell you based on the mood conveyed by the sound of your voice.</span> <span class="attribution"><a class="source" href="https://www.gettyimages.com/detail/illustration/woman-holding-red-phone-to-her-ear-royalty-free-illustration/186599890?adppopup=true">CSA-Printstock via Getty Images</a></span></figcaption></figure><p>You decide to call a store that sells some hiking boots you’re thinking of buying. As you dial in, the computer of an artificial intelligence company hired by the store is activated. It retrieves its analysis of the speaking style you used when you phoned other companies the software firm services. The computer has concluded you are “friendly and talkative.” Using predictive routing, it connects you to a customer service agent who company research has identified as being especially good at getting friendly and talkative customers to buy more expensive versions of the goods they’re considering.</p>
<p>This hypothetical situation may sound as if it’s from some distant future. But automated voice-guided marketing activities like this <a href="https://callminer.com/solutions/business-value/customer-experience/">are happening all the time</a>. </p>
<p>If you hear “This call is being recorded for training and quality control,” it isn’t just the customer service representative they’re monitoring. </p>
<p>It can be you, too. </p>
<p>When conducting research for my forthcoming book, “<a href="https://yalebooks.yale.edu/book/9780300248036/voice-catchers">The Voice Catchers: How Marketers Listen In to Exploit Your Feelings, Your Privacy, and Your Wallet</a>,” I went through over 1,000 trade magazine and news articles on the companies connected to various forms of voice profiling. I examined hundreds of pages of U.S. and EU laws applying to biometric surveillance. I analyzed dozens of patents. And because so much about this industry is evolving, I spoke to 43 people who are working to shape it.</p>
<p>It soon became clear to me that we’re in the early stages of a voice-profiling revolution that companies see as integral to the future of marketing. </p>
<p>Thanks to the public’s embrace of smart speakers, intelligent car displays and voice-responsive phones – along with the rise of voice intelligence in call centers – marketers say they are on the verge of being able to use AI-assisted vocal analysis technology to achieve unprecedented insights into shoppers’ identities and inclinations. In doing so, they believe they’ll be able to circumvent the errors and fraud associated with traditional targeted advertising.</p>
<p>Not only can people be profiled by their speech patterns, but they can also be assessed by the sound of their voices – which, <a href="https://books.google.com/books/about/Profiling_Humans_from_their_Voice.html?id=9_edDwAAQBAJ">according to some researchers</a>, is unique and can reveal their feelings, personalities and even their physical characteristics. </p>
<h2>Flaws in targeted advertising</h2>
<p>Top marketing executives I interviewed said that they expect their customer interactions to include voice profiling within a decade or so.</p>
<p>Part of what attracts them to this new technology is a belief that the current digital system of creating unique customer profiles – and then targeting them with personalized messages, offers and ads – <a href="https://som.yale.edu/blog/the-risks-and-rewards-of-targeted-ads">has major drawbacks</a>.</p>
<p>A simmering worry among internet advertisers, <a href="https://yalebooks.yale.edu/book/9780300248036/voice-catchers">one that burst into the open during the 2010s</a>, is that customer data often isn’t up to date, profiles may be based on multiple users of a device, names can be confused and people lie.</p>
<p>Advertisers are also uneasy about <a href="https://blog.nativeadvertisinginstitute.com/ad-blocking-impact-online-advertising-ecosystem">ad blocking</a> and <a href="https://www.cloudflare.com/learning/bots/what-is-click-fraud/">click fraud</a>, which happens when a site or app uses bots or low-paid workers to click on ads placed there so that the advertisers have to pay up. </p>
<p>These are all barriers to understanding individual shoppers. </p>
<p>Voice analysis, on the other hand, is seen as a solution that makes it nearly impossible for people to hide their feelings or evade their identities. </p>
<h2>Building out the infrastructure</h2>
<p>Most of the activity in voice profiling is happening in customer support centers, which are largely out of the public eye.</p>
<p>But there are also <a href="https://voicebot.ai/2020/01/06/amazon-now-claims-hundreds-of-millions-of-alexa-enabled-devices-and-hundreds-of-millions-of-weekly-smart-home-interactions/">hundreds of millions</a> of Amazon Echoes, Google Nests and other smart speakers out there. Smartphones also contain such technology.</p>
<p>All are listening and capturing people’s individual voices. They respond to your requests. But the assistants are also tied to advanced machine learning and deep neural network programs <a href="https://www.fastcompany.com/90409535/little-by-little-amazon-is-giving-alexa-more-ai-smarts">that analyze what you say and how you say it</a></p>
<figure class="align-center ">
<img alt="A cyborg wears a headset." src="https://images.theconversation.com/files/397432/original/file-20210427-13-w2bcy2.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=754&fit=clip" srcset="https://images.theconversation.com/files/397432/original/file-20210427-13-w2bcy2.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=600&h=530&fit=crop&dpr=1 600w, https://images.theconversation.com/files/397432/original/file-20210427-13-w2bcy2.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=600&h=530&fit=crop&dpr=2 1200w, https://images.theconversation.com/files/397432/original/file-20210427-13-w2bcy2.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=600&h=530&fit=crop&dpr=3 1800w, https://images.theconversation.com/files/397432/original/file-20210427-13-w2bcy2.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=754&h=666&fit=crop&dpr=1 754w, https://images.theconversation.com/files/397432/original/file-20210427-13-w2bcy2.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=754&h=666&fit=crop&dpr=2 1508w, https://images.theconversation.com/files/397432/original/file-20210427-13-w2bcy2.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=754&h=666&fit=crop&dpr=3 2262w" sizes="(min-width: 1466px) 754px, (max-width: 599px) 100vw, (min-width: 600px) 600px, 237px">
<figcaption>
<span class="caption">Call centers can use AI-assisted voice technology to determine whether to upsell certain customers.</span>
<span class="attribution"><a class="source" href="https://www.gettyimages.com/detail/illustration/person-wearing-a-telephone-headset-royalty-free-illustration/76185347?adppopup=true">Ralf Hiemisch via Getty Images</a></span>
</figcaption>
</figure>
<p>Amazon and Google – the leading purveyors of smart speakers outside China – appear to be doing little voice analysis on those devices beyond recognizing and responding to individual owners. Perhaps they fear that pushing the technology too far will, at this point, lead to bad publicity.</p>
<p>Nevertheless, the user agreements of Amazon and Google – as well as Pandora, Bank of America and other companies that people access routinely via phone apps – give them the right to use their digital assistants <a href="https://yalebooks.yale.edu/book/9780300248036/voice-catchers">to understand you by the way you sound</a>. Amazon’s most public application of voice profiling so far is its Halo wristband, <a href="https://www.bloomberg.com/news/newsletters/2020-08-31/amazon-s-halo-wearable-can-read-emotions-is-that-too-weird">which claims to know the emotions you’re conveying</a> when you talk to relatives, friends and employers.</p>
<p><a href="https://www.amazon.com/gp/help/customer/display.html/ref=hp_bc_nav?ie=UTF8&nodeId=GL99TQL4B7ADPBDH">The company assures customers it doesn’t use Halo data for its own purposes</a>. But it’s clearly a proof of concept – and a nod toward the future.</p>
<h2>Patents point to the future</h2>
<p>The patents from these tech companies offer a vision of what’s coming.</p>
<p><a href="https://patents.google.com/patent/US10096319B1/en">In one Amazon patent</a>, a device with the Alexa assistant picks up a woman’s speech irregularities that imply a cold through using “an analysis of pitch, pulse, voicing, jittering, and/or harmonicity of a user’s voice, as determined from processing the voice data.” From that conclusion, Alexa asks if the woman wants a recipe for chicken soup. When she says no, it offers to sell her cough drops with one-hour delivery.</p>
<figure class="align-center zoomable">
<a href="https://images.theconversation.com/files/397139/original/file-20210426-13-9ahkdi.png?ixlib=rb-1.1.0&q=45&auto=format&w=1000&fit=clip"><img alt="A page from an Amazon patent depicts a woman interacting with a home assistant." src="https://images.theconversation.com/files/397139/original/file-20210426-13-9ahkdi.png?ixlib=rb-1.1.0&q=45&auto=format&w=754&fit=clip" srcset="https://images.theconversation.com/files/397139/original/file-20210426-13-9ahkdi.png?ixlib=rb-1.1.0&q=45&auto=format&w=600&h=387&fit=crop&dpr=1 600w, https://images.theconversation.com/files/397139/original/file-20210426-13-9ahkdi.png?ixlib=rb-1.1.0&q=30&auto=format&w=600&h=387&fit=crop&dpr=2 1200w, https://images.theconversation.com/files/397139/original/file-20210426-13-9ahkdi.png?ixlib=rb-1.1.0&q=15&auto=format&w=600&h=387&fit=crop&dpr=3 1800w, https://images.theconversation.com/files/397139/original/file-20210426-13-9ahkdi.png?ixlib=rb-1.1.0&q=45&auto=format&w=754&h=486&fit=crop&dpr=1 754w, https://images.theconversation.com/files/397139/original/file-20210426-13-9ahkdi.png?ixlib=rb-1.1.0&q=30&auto=format&w=754&h=486&fit=crop&dpr=2 1508w, https://images.theconversation.com/files/397139/original/file-20210426-13-9ahkdi.png?ixlib=rb-1.1.0&q=15&auto=format&w=754&h=486&fit=crop&dpr=3 2262w" sizes="(min-width: 1466px) 754px, (max-width: 599px) 100vw, (min-width: 600px) 600px, 237px"></a>
<figcaption>
<span class="caption">An Amazon patent depicts a device picking up a woman’s cough – and then asking if she wants a recipe for chicken soup.</span>
<span class="attribution"><a class="source" href="https://patents.google.com/patent/US10096319B1/en">Google Patents</a></span>
</figcaption>
</figure>
<p><a href="https://patents.justia.com/patent/10262661">Another Amazon patent</a> suggests an app to help a store salesperson decipher a shopper’s voice to plumb unconscious reactions to products. The contention is that how people sound allegedly does a better job indicating what people like than their words.</p>
<p><a href="https://patents.google.com/patent/US20160259308A1/en">And one of Google’s proprietary inventions</a> involves tracking family members in real time using special microphones placed throughout a home. Based on the pitch of voice signatures, Google circuitry infers gender and age information – for example, one adult male and one female child – and tags them as separate individuals. </p>
<p>The company’s patent asserts that over time the system’s “household policy manager” will be able to compare life patterns, such as when and how long family members eat meals, how long the children watch television, and when electronic game devices are working – and then have the system suggest better eating schedules for the kids, or offer to control their TV viewing and game playing.</p>
<h2>Seductive surveillance</h2>
<p>In the West, the road to this advertising future starts with firms encouraging users to give them permission to gather voice data. Firms gain customers’ permission by enticing them to buy inexpensive voice technologies.</p>
<p>When tech companies have further developed voice analysis software – and people have become increasingly reliant on voice devices – I expect the companies to begin widespread profiling and marketing based on voice data. Hewing to the letter if not the spirit of whatever privacy laws exist, the companies will, I expect, forge ahead into their new incarnations, even if most of their users joined before this new business model existed.</p>
<p><a href="https://hoofnagle.berkeley.edu/2018/02/19/bait-and-switch-advertising-bait-and-switch-privacy/">This classic bait and switch marked the rise of both Google and Facebook</a>. Only when the numbers of people flocking to these sites became large enough to attract high-paying advertisers did their business models solidify around selling ads personalized to what Google and Facebook knew about their users. </p>
<p>By then, <a href="https://www.washingtonpost.com/technology/2020/11/19/can-not-quit-facebook/">the sites had become such important parts of their users’ daily activities that people felt they couldn’t leave</a>, despite their concerns about data collection and analysis that they didn’t understand and couldn’t control. </p>
<p>This strategy is already starting to play out as tens of millions of consumers <a href="https://onezero.medium.com/amazon-and-google-are-practically-giving-away-smart-speakers-heres-why-56f0e50bd95c">buy Amazon Echoes at giveaway prices</a>.</p>
<p>[<em>Insight, in your inbox each day.</em> <a href="https://theconversation.com/us/newsletters/the-daily-3?utm_source=TCUS&utm_medium=inline-link&utm_campaign=newsletter-text&utm_content=insight">You can get it with The Conversation’s email newsletter</a>.]</p>
<h2>The dark side of voice profiling</h2>
<p>Here’s the catch: It’s not clear how accurate voice profiling is, especially when it comes to emotions.</p>
<p>It is true, <a href="https://www.springerprofessional.de/en/profiling-humans-from-their-voice/16897712#:%7E:text=The%20term%20profiling%20from%20voice,in%20the%20human%20vocal%20tract">according to Carnegie Mellon voice recognition scholar Rita Singh</a>, that the activity of your vocal nerves is connected to your emotional state. However, Singh told me that she worries that with the easy availability of machine-learning packages, people with limited skills will be tempted to run shoddy analyses of people’s voices, leading to conclusions that are as dubious as the methods. </p>
<p>She also argues that inferences that link physiology to emotions and forms of stress may be culturally biased and prone to error. That concern hasn’t deterred marketers, who typically use voice profiling to draw conclusions about individuals’ emotions, attitudes and personalities.</p>
<p>While some of these advances <a href="https://yalebooks.yale.edu/book/9780300248036/voice-catchers">promise to make life easier</a>, it’s not difficult to see how voice technology can be abused and exploited. What if voice profiling tells a prospective employer that you’re a bad risk for a job that you covet or desperately need? What if it tells a bank that you’re a bad risk for a loan? What if a restaurant decides it won’t take your reservation because you sound low class, or too demanding? </p>
<p>Consider, too, the discrimination that can take place <a href="https://books.google.com/books/about/Profiling_Humans_from_their_Voice.html?id=9_edDwAAQBAJ">if voice profilers follow some scientists’ claims</a> that it is possible to use an individual’s vocalizations to tell the person’s height, weight, race, gender and health.</p>
<p>People are already subjected to different offers and opportunities based on the personal information companies have collected. Voice profiling adds an especially insidious means of labeling. Today, some states such as Illinois and Texas <a href="https://www.thompsonhine.com/publications/state-biometric-privacy-legislation-what-you-need-to-know">require companies to ask for permission</a> before conducting analysis of vocal, facial or other biometric features. </p>
<p>But other states expect people to be aware of the information that’s collected about them from the privacy policies or terms of service – <a href="https://theconversation.com/nobody-reads-privacy-policies-heres-how-to-fix-that-81932">which means they rarely will</a>. And the federal government hasn’t enacted a sweeping marketing surveillance law.</p>
<p>With the looming widespread adoption of voice analysis technology, it’s important for government leaders to adopt policies and regulations that protect the personal information revealed by the sound of a person’s voice.</p>
<p>One proposal: While the use of <a href="https://www.techopedia.com/definition/13707/voice-authentication">voice authentication</a> – or using a person’s voice to prove their identity – could be allowed under certain carefully regulated circumstances, all voice profiling should be prohibited in marketers’ interactions with individuals. This prohibition should also apply to political campaigns and to government activities without a warrant. </p>
<p>That seems like the best way to ensure that the coming era of voice profiling is constrained before it becomes too integrated into daily life and too pervasive to control.</p><img src="https://counter.theconversation.com/content/158921/count.gif" alt="The Conversation" width="1" height="1" />
<p class="fine-print"><em><span>Joseph Turow does not work for, consult, own shares in or receive funding from any company or organization that would benefit from this article, and has disclosed no relevant affiliations beyond their academic appointment.</span></em></p>Marketers will soon be able to use AI-assisted vocal analysis to gain insights into shoppers’ inclinations – without people knowing what they’re revealing or how that information is being interpreted.Joseph Turow, Robert Lewis Shayon Professor of Media Systems & Industries, University of PennsylvaniaLicensed as Creative Commons – attribution, no derivatives.tag:theconversation.com,2011:article/1457812020-09-08T15:08:29Z2020-09-08T15:08:29ZSmart speakers: why sales are rocketing despite all our privacy fears<figure><img src="https://images.theconversation.com/files/356996/original/file-20200908-24-1o67exu.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=496&fit=clip" /><figcaption><span class="caption">Opportunity knocks. </span> <span class="attribution"><a class="source" href="https://www.flickr.com/photos/182958824@N08/48369722906/in/photolist-2gGgtrL-2gGgtq3-2gGgto9-2cvZdBp-C35bQ4-b5TM7e-2hSbwUL-A72omm-T11wMo-A72opN-AHNt2Y-A72qUB-A9kAjT-AHNsV5-eD9xu4-2cyw6n4-eDfEm7-w9HzSH-AGGfyJ-2dRtKdv-2iQ4EwC-2iPZ2Fh-2iPYUZw-2iQ2C3T-2iPsJ5e-2iPr3k5-2iNYs3R-2iNwS2D-2iNu7bZ-2iPZ4dv-2iQ4roZ-2iPr5Fn-2iPsJik-2iNwYBh-2iNucup-2iNubHz-2iNwThe-2iNwM6M-2iNwLrL-2iNwKx6-2iNtZqz-2iNynio-2iNymWg-2iNwuM4-2iNwtFG-2iNtHAJ-2iNu9WD-9KY5B6-9jAPYu-o6dE6s">Eshop Citylink</a></span></figcaption></figure><p>With everyone spending so much time at home during the pandemic, smart speakers such as the Amazon Echo and Google Nest ranges have had a golden opportunity. In their latest attempt to make the devices as relevant as possible to this captive audience, various device makers <a href="https://www.theverge.com/2020/8/19/21373661/zoom-smart-displays-facebook-portal-google-nest-amazon-echo-show">recently announced</a> that they are incorporating Zoom videoconferencing capabilities into speakers fitted with screens.</p>
<p>One seemingly big obstacle to all this is the fact that consumers have been increasingly worried about tech privacy in recent years. There <a href="https://www.theguardian.com/technology/2019/oct/09/alexa-are-you-invading-my-privacy-the-dark-side-of-our-voice-assistants">have been</a> many <a href="https://www.nbcnews.com/tech/innovation/alexa-privacy-fail-highlights-risks-smart-speakers-n877671">media stories</a> about what smart speakers can capture and share. According to <a href="https://www.capgemini.com/research/smart-talk/">one major survey</a> in 2019, half of all respondents said they don’t trust voice assistants with the safety and security of their personal data. </p>
<p>So what has that meant for sales of smart speakers? In response to all these cheap devices, how many of us are saying, “Alexa, leave us alone”?</p>
<h2>Privacy issues …</h2>
<p>A product category that didn’t exist until late in 2014 with the <a href="https://www.theverge.com/2014/11/6/7167793/amazon-echo-speaker-announced">launch of</a> the first Amazon Echo, these devices are essentially the interface for voice-driven virtual assistants like Amazon’s Alexa, Google’s Assistant, Apple’s Siri and Baidu’s DuerOS. Often you can get the same functionality from a built-in assistant in your smartphone or tablet, but smart speakers and screens give this capability on a standing device at home. </p>
<p>Privacy <a href="https://www.theguardian.com/technology/2015/nov/21/amazon-echo-alexa-home-robot-privacy-cloud">fears have</a> been <a href="https://www.washingtonpost.com/gdpr-consent/?next_url=https%3a%2f%2fwww.washingtonpost.com%2fnews%2fthe-switch%2fwp%2f2014%2f11%2f11%2fhow-closely-is-amazons-echo-listening%2f">around since</a> the early days. “Always listening” microphones concerned people, even though the device makers promised they would only listen if someone spoke directly to the device via its “wake word” or phrase.</p>
<p>Yet we know that smart speakers can <a href="https://www.howtogeek.com/427686/how-alexa-listens-for-wake-words/">always be listening</a>, and can <a href="https://www.consumerreports.org/smart-speakers/smart-speakers-that-listen-when-they-shouldnt/">wake up and record</a> more than people realise. The supposed protection from the “wake word” can easily be changed by an errant software update – for instance, a recent mistake in pushing out a software update to too many users <a href="https://www.protocol.com/google-smart-speaker-alarm-adt">led to</a> Google smart speakers waking based on other sounds in the home (the update has since been recalled).</p>
<p>Recordings from devices have been streamed out of the home to the manufacturer, where staff have reportedly <a href="https://www.vrt.be/vrtnws/en/2019/07/10/google-employees-are-eavesdropping-even-in-flemish-living-rooms/">listened to them</a> to try to make the voice recognition better. In one reported case, 1,000 of these recordings <a href="https://mashable.com/article/google-assistant-recordings-leaked/?europe=true">were leaked</a>.</p>
<figure class="align-center zoomable">
<a href="https://images.theconversation.com/files/356997/original/file-20200908-16-rl7ul4.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=1000&fit=clip"><img alt="Google Nest Mini" src="https://images.theconversation.com/files/356997/original/file-20200908-16-rl7ul4.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=754&fit=clip" srcset="https://images.theconversation.com/files/356997/original/file-20200908-16-rl7ul4.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=600&h=483&fit=crop&dpr=1 600w, https://images.theconversation.com/files/356997/original/file-20200908-16-rl7ul4.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=600&h=483&fit=crop&dpr=2 1200w, https://images.theconversation.com/files/356997/original/file-20200908-16-rl7ul4.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=600&h=483&fit=crop&dpr=3 1800w, https://images.theconversation.com/files/356997/original/file-20200908-16-rl7ul4.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=754&h=607&fit=crop&dpr=1 754w, https://images.theconversation.com/files/356997/original/file-20200908-16-rl7ul4.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=754&h=607&fit=crop&dpr=2 1508w, https://images.theconversation.com/files/356997/original/file-20200908-16-rl7ul4.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=754&h=607&fit=crop&dpr=3 2262w" sizes="(min-width: 1466px) 754px, (max-width: 599px) 100vw, (min-width: 600px) 600px, 237px"></a>
<figcaption>
<span class="caption"></span>
<span class="attribution"><a class="source" href="https://www.flickr.com/photos/13815526@N02/11951299695/in/photolist-jd6x2i-2iqCRJ5-21TLCCf-jd826R-TdQNoQ-CpWD4y-jd6wUK-2iNwUiH-vGu8oH-8rdTVT-2iNua9n-261Ntn4-T2h3Dp-rdyoBp-261Nurt-G5NuCK-wDQVVt-TM6Rky-261NuRg-qS2Aq2-HJqovR-261NtBH-2iNukuC-2iNyBxA-HJqqvT-261Nu1t-HJqoKt-HJqkeR-261NuAr-REtVTh-8ojWKi-ej9V4m-uiTnyQ-2j67yVW-9S7QCL-KHiU1G-T1xSuE-FZJLgJ-LykQQ4-2j64Vue-Uab8X6-TREWJo-Tbb2WD-TB2bSy-SB8oU6-pNxfeB-XTuq7G-2dH9Jc5-SPT5Ac-2gZf4yk">Pierre Lecourt</a>, <a class="license" href="http://creativecommons.org/licenses/by-sa/4.0/">CC BY-SA</a></span>
</figcaption>
</figure>
<p>In a <a href="https://www.pwc.com/us/en/advisory-services/publications/consumer-intelligence-series/voice-assistants.pdf">2018 American survey</a>, 38% of people said they didn’t want something “listening in” on their life all the time, and 28% were concerned about privacy issues. In the <a href="https://www.capgemini.com/research/smart-talk/">2019 survey</a> I mentioned earlier, which included consumers in the US, UK, France and Germany, 52% were worried about “passive” always-listening smart speakers and voice assistants, and the risk of them listening to private conversations. <a href="https://www.ipsos.com/sites/default/files/ct/publication/documents/2019-03/techtracker_report_q12019_final_1.pdf">A separate study</a> also reached the same conclusion: 52% of UK customers have concerns about privacy.</p>
<p>It’s not just customers who are concerned about privacy – some businesses have spoken up. The international law firm Mishcon de Reya asked all its staff working from home to <a href="https://www.bloomberg.com/news/articles/2020-03-20/locked-down-lawyers-warned-alexa-is-hearing-confidential-calls">mute or turn off</a> smart speakers, and any other visual or voice-enabled devices, when talking about client matters at home.</p>
<h2>… and record sales</h2>
<p>In spite of all these concerns, <a href="https://nypost.com/2019/12/26/smart-speakers-were-some-of-the-holiday-seasons-top-stocking-stuffers-in-2019/">smart speakers were</a> one of the most popular Christmas gifts of 2019, capping a record year for sales. Worldwide, 147 million devices <a href="https://techcrunch.com/2020/02/17/smart-speaker-sales-reached-new-record-of-146-9m-in-2019-up-70-from-2018/">were shifted</a> across the year, a 70% increase on 2018. A fair proportion was in China as Baidu, Alibaba and Xiaomi all made headway with their own smart devices: they are now the third, fourth and fifth biggest vendors after Amazon (first with 22%) and Google (second with 17%). </p>
<p>During the pandemic, sales <a href="https://www.businesswire.com/news/home/20200811005706/en/Strategy-Analytics-Global-Smart-Speaker-Sales-Rose">have held up</a> very well. China had a weaker first quarter and the US and Europe had a weaker second quarter, but world smart-speaker sales for the year are expected to be about 161 million – 10% up year on year, despite the trough in the global economy. </p>
<figure class="align-right zoomable">
<a href="https://images.theconversation.com/files/356998/original/file-20200908-24-1di9cej.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=1000&fit=clip"><img alt="Baidu Smart Ai Speaker Pro" src="https://images.theconversation.com/files/356998/original/file-20200908-24-1di9cej.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=237&fit=clip" srcset="https://images.theconversation.com/files/356998/original/file-20200908-24-1di9cej.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=600&h=800&fit=crop&dpr=1 600w, https://images.theconversation.com/files/356998/original/file-20200908-24-1di9cej.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=600&h=800&fit=crop&dpr=2 1200w, https://images.theconversation.com/files/356998/original/file-20200908-24-1di9cej.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=600&h=800&fit=crop&dpr=3 1800w, https://images.theconversation.com/files/356998/original/file-20200908-24-1di9cej.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=754&h=1005&fit=crop&dpr=1 754w, https://images.theconversation.com/files/356998/original/file-20200908-24-1di9cej.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=754&h=1005&fit=crop&dpr=2 1508w, https://images.theconversation.com/files/356998/original/file-20200908-24-1di9cej.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=754&h=1005&fit=crop&dpr=3 2262w" sizes="(min-width: 1466px) 754px, (max-width: 599px) 100vw, (min-width: 600px) 600px, 237px"></a>
<figcaption>
<span class="caption">Baidu: the Chinese #1.</span>
<span class="attribution"><a class="source" href="https://www.shutterstock.com/image-photo/nanjing-china-november-4-2019-baidu-1551298196">abolukbas</a></span>
</figcaption>
</figure>
<p>With <a href="https://voicebot.ai/2020/04/28/nearly-90-million-u-s-adults-have-smart-speakers-adoption-now-exceeds-one-third-of-consumers/">the US</a> and <a href="https://www.statista.com/statistics/1107880/access-to-smart-speaker-in-households-worldwide/#:%7E:text=Access%20to%20a%20smart%20speaker%20in%20households%20worldwide%202020&text=According%20to%20the%20survey%2C%20in,smart%20speaker%20in%20their%20households">UK</a> at the forefront in terms of penetration, around one in three people now has access to a device in those markets. <a href="https://www.statista.com/statistics/946437/china-share-of-smart-speaker-users-among-internet-users/">In China</a>, it is about one in ten of those with internet access. </p>
<p>So how to reconcile these stats with the privacy fears? One clue is that the most common reason why people in the UK reported using a smart speaker was because they received it as a gift – <a href="https://www.ipsos.com/sites/default/files/ct/publication/documents/2019-03/techtracker_report_q12019_final_1.pdf">more than half did so</a>. The next most common (and growing) reason was music streaming.</p>
<p>We also know <a href="https://www.diva-portal.org/smash/get/diva2:1096364/FULLTEXT01.pdf">from research</a> that people find it less convenient to press a button to wake a speaker than to activate it by voice. So it may be that people are caught between wanting privacy on the one hand but wanting convenience at the same time. </p>
<p>Smart speakers make us more loyal to the tech giants that control them. <a href="https://www.audienceproject.com/blog/key-insights/new-study-spotify-crushes-the-tech-giants/">For example</a>, Spotify has a market share of around 49% in the US, but <a href="https://www.nationalpublicmedia.com/uploads/2020/01/The-Smart-Audio-Report-Winter-2019.pdf">only accounted for</a> 20% of audio streaming time on smart speakers; Amazon Music, on the other hand, makes up 33%. Presumably it benefits from being the <a href="https://bigtechquestion.com/2020/01/06/smarthome/amazonalexa/why-wont-alexa-play-my-spotify-playlist/">default option</a> on Echo devices. </p>
<p>Some other smart speakers give even less control of this – Apple’s HomePod speaker has <a href="https://www.macrumors.com/2020/07/07/homepod-beta-2-default-services-option/">only recently</a> made available an option to choose the default music service for audio, in a pre-release version of the software. Seemingly beaten by these gatekeepers, Spotify <a href="https://metro.co.uk/2020/09/02/how-to-get-free-google-mini-nest-spotify-13212399/">has done a deal</a> with Google to give away Google Nest Mini devices to its premium subscribers to try and bounce back. </p>
<p>Whether or not this succeeds, it demonstrates that the key opportunity afforded by smart speakers – and the growing range of other smart home technologies – is to provide device makers with an avenue to make it easier for customers to use their other products and services. Having apparently weathered people’s privacy concerns, the companies that rule this frontier look poised to satisfy an ever greater share of consumer needs in future.</p><img src="https://counter.theconversation.com/content/145781/count.gif" alt="The Conversation" width="1" height="1" />
<p class="fine-print"><em><span>Greig Paul does not work for, consult, own shares in or receive funding from any company or organisation that would benefit from this article, and has disclosed no relevant affiliations beyond their academic appointment.</span></em></p>“Alexa, why are the likes of Google and Amazon so determined to get us all to buy smart speakers?”Greig Paul, Lead Mobile Networks and Security Engineer, University of Strathclyde Licensed as Creative Commons – attribution, no derivatives.