UK United Kingdom

Facebook and facial recognition – you’ve been tagged

Around 2 million photographs are uploaded to Facebook each day. As of this week, every new image will be processed by automatic face recognition software, designed to identify the people in the photographs…

Your Facebook snaps now come with a hidden catch. rishibando

Around 2 million photographs are uploaded to Facebook each day. As of this week, every new image will be processed by automatic face recognition software, designed to identify the people in the photographs.

It’s not the addition of this new function that has caused the creeping sense of unease, but the covert manner with which it has been activated.

The new feature has been switched on as a default: all of your photos will now be processed and tagged unless you decide to opt out of the service.

What should have been provided as a tool to make lives easier has instead left people feeling their personal data is under the control of a technology they don’t understand.

How worried should we be about the widespread implementation of face recognition technology?

First of all it is important to understand that the technology would not work without the Facebook user’s guidance. As Dr Brian Lovell has already explained to the Australian media this week, face recognition software is especially poor when dealing with images taken in uncontrolled environments such as a bar, club, and music festival locations, which appear to make up the majority of the 90 billion image corpus.

Given the significant degree of error generated by these algorithms (my colleague Dr Kristy Martire told me today that her friend was tagged as an obscure Argentinian statue), the user input is necessary to constrain the number crunching.

Manual tagging is actually the clever part, because humans have an especially impressive ability to recognise faces. Psychologists generally agree this is the most advanced feat of recognition that the human visual system has developed.

To understand just how good we are at recognising familiar faces it’s useful to firstly consider just how difficult the task becomes with unfamiliar faces. So, a little experiment:

In the image array below I have collated ten photographs. The one you see on the bottom right is my current passport image. Five are from my own Facebook account. The remaining four are of a friend that looks similar to me.

I invite you to try and cleave this mixture of images into two separate identities. Depending on how confident you feel, you can post your answer in the comments section. If you do, I will pick the winner later.

Psychological research has repeatedly shown that humans are actually very bad at verifying the identity of unfamiliar people from photographs. This is true when comparing good quality photographs to one another, and also when comparing photo-ID to physically present individuals.

Because the same tasks become trivially easy when we are familiar with the person in the images, a number of visual scientists are currently trying to unravel the process that drives this improvement.

Computers and people are similar in respect to faces – both are unable to reliably tell if two images of unfamiliar people are of the same person or two different people. This unreliability is a real problem in security settings and perhaps why it has been deemed necessary to allow other biometric data (such as iris recognition) to be stored on travel documents.

Civil libertarians beware, however: Facebook provides the ideal platform to develop a procedure for accurate automatic facial recognition.

Firstly, it harnesses humans expertise at recognising familiar faces by giving them a monitoring role.

Secondly, the high degree of image variation available on user profiles is in fact ideal for training face recognition algorithms, allowing them to reach an almost perfect level of accuracy when searching small databases.

The performance of face recognition software has improved dramatically over the last ten years. This improvement leads some to believe that, instead of having to endure all the boring and time-consuming introductions at social gatherings, we might soon be able to shove our camera phone in the faces of strangers and be done with it.

Alas, current data suggest this will not happen soon, if at all.

In the meantime, it’s not the algorithms that we should worry about. It is the snap-happy friend who indiscriminately tags photos of you in your least flattering pose.

What do you think about the rollout of this technology by Facebook? Can you identify who is who in the above experiment? Leave your comments (and answers) below.

Sign in to Favourite

Join the conversation

12 Comments sorted by

  1. Paul Dalgarno

    Editor at The Conversation

    I'm actually really struggling with the test ... to my shame ... hoping someone out there is better than me. Lunch with an editor of your choice for the winner (provided you pay) ...

    1. David White

      Postdoctoral Research Fellow at UNSW Australia

      In reply to Paul Dalgarno

      You're too generous Paul- I will post the answer on Monday...

      Answers coded as A & B please - i.e. if you think the top row is him and the bottom row is me then BBBBBAAAAA

  2. Ashley Anderson

    Ashley Anderson

    Whoa. What if you are not a member of Facebook, and have not given them permission to store your face? I have notice that people can manually tag me, even though I don't have an account; I would be really upset if Facebook itself could tag me!

  3. Marie-A Lo Cascio

    logged in via Twitter

    Interesting article and what it Ashley points out being tagged by others is a real worry!
    For what it's worth I think it's the same person in all the photos.

  4. Justin Browne


    Interesting article.

    I found the facial identification challenge quite difficult. My answers: ABAAA BBABA.

    (Which I now see is the same order as Ashley's).

  5. Jan Tilden

    Science communicator

    Yep. That would probably be my first pick. The last two are the hardest. Just to be a little different, I'll opt for ABAAA BBABB

    1. Jan Tilden

      Science communicator

      In reply to Jan Tilden

      Should have read the story properly and looked at your profile photo. I agree with the others: ABAAA BBABA.

  6. David White

    Postdoctoral Research Fellow at UNSW Australia

    Thanks to those of you that had a go at the test.

    It seems that I underestimated people’s ability to perform this task as you almost all got it right (ABAAA BBABA)!

    Perhaps I made it too easy by telling you that there were two people in the array, it is much more difficult when you don’t know how many you are looking for. Incidentally, a computer's performance on this test would entirely depend on where a human had set the 'False Acceptance Rate'.

    Over the weekend I noticed an inaccuracy in the article. I start the piece by saying that 2 million images are uploaded each day, but came across an interview with Mark Zuckerberg where he says that this figure is actually 100 million! I assume that means that the 90 billion figure is also much greater.

    This really is a staggering amount of data, and I think Ashley and others are justifiably wary.

  7. Filippo Mammana

    logged in via Facebook

    I believe all the images are of only 1 male. Am I correct?

  8. Stephen Wilson


    Personal Information is gold in the digital economy. At its core, Facebook offers an unfair bargain for its members' PI. In return for all the fun and games, members innocently gift to Facebook their demographic data, their connections, their express likes & dislikes, and through their behaviour, their tacit preferences too. Moreover, Facebook are breathtakingly good at manufacturing PI, and with facial recogniton they are taking this to new levels. They are literally printing money!

    When Facebook…

    Read more