“Move fast and break things.” That is the Facebook motto plastered all over their California headquarters to remind engineers never to stop innovating. This week, the company moved fast and broke some news by investing in a thing called machine learning. They appointed two professors, Yann LeCun and Rob Fergus, from New York University, to lead new artificial intelligence labs in New York, California and London.
Breakthroughs in artificial intelligence (AI) are rare, but last year a team led by Geoff Hinton at the University of Toronto made a significant step forward in understanding the content of images. Geoff and his team set up a company, DNNresearch, to exploit the advance, and were quickly purchased by Google. The technology has already been used to advance image search in Google+. Users can now search their photos for bicycles, dogs, beaches and sunsets as well as thousands of other categories.
The technology they used is known as “deep learning”. So the Facebook announcement was Mark Zuckerberg’s response to Google’s move. By involving leading academics he wants to make sure the job is done right. He attended the conference, hosting a small reception for some of the leading researchers the night before the public announcement. He spent the rest of his time recruiting researchers for the new AI labs.
The reason for the California lab is clear, but why New York and London? The labs are deliberately spread across the world to make it as easy as possible for the leading researchers to move: LeCun and Fergus’s new office is only a couple of blocks away from their current location in downtown Manhattan. Details of the London lab are still unclear.
Machine learning is not a new technology at Facebook: they use it to place adverts, rank news feed items and suggest friends for their users. In July 2012 Facebook flew in researchers from around the world to their Menlo Park headquarters for their first “Academic Summit”. But this new initiative has a different feel to it. It is not common for the CEO of one of the world’s largest companies to turn up at a 2,000-person conference on a large-scale recruitment drive. Facebook is targeting full-scale artificial intelligence, something that could compete with a human, and its bet is that deep learning is the right way to arrive at that goal.
Deep thinking
Deep learning technologies are inspired by the layout of neurons in the brain. The signals transmitted by our retina to a region in the brain called the visual cortex are processed by a set of layers of neurons before being interpreted. The first layers of neurons start by detecting simple features in the images, such as the edges of objects. Each layer, in turn, composes more of these features together until the parts of the object are formed. Finally, in the upper layers, a complete understanding of the visual scene can be composed and recognised by our brains.
Hinton and LeCun both began working on computer models more than 25 years ago, inspired by the brain’s “deep” architecture. What enables these models to be learned now is the wide availability of data for these models to learn from. This phenomenon, known as big data, is changing the way all businesses are run. Targeted advertising from supermarkets, movie recommendations from Netflix and book suggestions from Amazon are all driven by machine learning models which predict human behaviour.
These luminaries have been attending the NIPS conference for more than 25 years, and this latest breakthrough is just one of a number of technologies that has emerged across this time. Industry has taken note. As well as Facebook and Google the other leading US internet firms: Amazon, Yahoo and Microsoft were also present. News has also spread overseas, the Russian market leader Yandex and the dominant Chinese search provider, Baidu, also attended.
LeCun is on record as stating that the next big challenge for these models is understanding language. This would improve speech recognition and allow search engines to understand the sentiment of a sentence, rather than just counting the statistics of which words are present. Maybe then Facebook could understand your sarcastic status update.