All-seeing eye: the future of surveillance and social media

Knowing me, knowing you: there is nothing we can’t view. tonechootero/Flickr

Advanced surveillance and social media might seem like strange bedfellows. Until you look a bit closer, that is.

Technologies developed for surveillance applications are typically designed with robustness in mind: that is, they should work reliably at all times in a variety of lighting conditions (indoor/outdoor) and effects (glare, saturation or shadows).

Compare this to the task of recognition from a photographic portait. Here the professional photographer has positioned the camera, lighting and subject and then chosen the exact moment to click the shutter. The photographer takes dozens of photos and selects the best.

Finally, the subject is presented with a great portait shot they are happy to show the world.

Interestingly, “robustness” in a surveillance context can make surveillance technologies ideal for images and video in the social and mobile context. As mobile phone usage is growing, so too is the use of mobile phones for creating and sharing images and video content with others in online social networks.

Mobile video has the highest growth rate of any mobile application category, and is expected to more than double every year between 2010 and 2015. If such predictions prove to be true, by 2015 two-thirds of the world’s mobile data traffic will be video.

The same but different

Handheld mobile images and video content often suffer from quality issues similar to those faced in surveillance, including motion blur and lighting changes due to variation between scenes.

So what to do? One of the most applicable technologies to social media, and particularly mobile social media, is face-recognition, as discussed earlier in the series.

Face recognition has been gaining popularity both in the surveillance and social realms over the past few years. In surveillance, this technology is often used for border control, and a number of different organisations are trialling this technology for general security.

In social media, competition is heating up among the software giants with Google and Apple each acquiring their own mobile face recognition companies, Neven Vision and Polar Rose respectively.

These technologies may have been used in the companies’ photo management applications (Google’s Picasa and Apple’s iPhoto), but they have not yet reached their mobile platforms (Android and iOS respectively).

A number of other developers have also entered the fray:

  • Microsoft is developing its own face-recognition technology at OneVision, but it has not been released in any products so far.
  • Facebook recently released its own face-recognition technology for easy tagging of photos online, but has yet to extend it into its API or mobile app.
  • Several face-recognition start-ups, such as Viewdle and Face.com are also targeting the mobile space with a focus on Google Android platforms.

More than a just a pretty face

Face-recognition is only one example of the possible cross-over from surveillance analytics to social media: object-detection and classification may be applicable to photo tagging as well.

One example is Google Goggles, which allows users to recognise landmarks and corporate logos from their mobile phone and look them up on Wikipedia.

But the relationship between surveillance and social media is not a one-way street.

Technologies used and developed for social uses, such as 3D games and augmented reality, can be used to enhance surveillance applications. One example of such technology is an augmented reality system that overlays multiple, live video camera feeds onto a single, texture-mapped 3D model of the environment.

The system was orginally developed for the US military as an intuitive presentation layer that would make surveillance operators effective with just 10 minutes of training. Gone are the rows of TV monitors: surveillance systems can now use a single screen and resemble a game of Doom.

Such 3D immersive surveillance systems are a key technology used by the Advanced Surveillance Project in NICTA.

At the moment, virtual models need to be constructed for every facility under surveillance, a situation that’s expensive to create and maintain. But developments such as Google Street View and Bing Maps’ street panoramas may allow the 3D texture maps to be available publicly in the same way that OpenStreetMap is now providing map data for navigation.

The surveillance industry is currently undergoing the same revolutionary changes that shook up the computer industry when internet use took off in the 1990s. Instead of each supplier providing a unique product, the sector will soon be dominated by standards and interoperability.

Surveillance will eventually merge into a virtually seamless multimedia network embracing social media, location services, mobile devices, maps, and 3D models.

It is likely every one of us, in some form or another, will have access to the advanced surveillance technologies being developed in the here and now, and that these technologies, in some way, will have access to us.

This is the fifth and final part of our series on advanced surveillance. To read the first four instalments, follow the links below: