Xbf9pcrk 1474448345

We made an app to identify bird sounds – and learned something surprising about people

We made an app to identify bird sounds – and learned something surprising about people

The sound of birds is everywhere but what if you can’t tell your robin from your ring-necked parrot? And when the birds listen to each other, are they picking up on the same qualities as we are? As part of my research, I try to help answer these questions by developing techniques for computers to make sense of sounds automatically, for example to identify a bird from its sound.

You don’t realise quite how amazing the human brain is at understanding sound until you’ve tried to “explain” the process to a computer so that it can do it on its own. In my lab at Queen Mary University of London we came up with an algorithm to dig through many thousands of bird recordings from hundreds of different bird species, and then to “teach” the computer to recognise common dimensions of variation in the sound. We used this to identify the species of birds in the rainforest and we got great results.

We then decided to make our algorithm into a smartphone app, this time trained to recognise birds in Britain, so that people around the country could record and identify birdsong. The surprise came when we heard what sounds they actually recorded – an oddly funny set of bird impressions. We ended up learning something interesting about humans as well as birds.

Smartphones and bird song

When you’re conducting research, it’s always useful to have plenty of data. Our work is based on machine learning, the type of artificial intelligence that involves software adapting to large amounts of data so that it can perform a certain way without being specifically programmed to. Large datasets are a crucial part of the technology that allows smartphones to recognise voice commands, for example.

Ecologists and other biologists are also starting to find themselves awash with audio and video data, since they can now easily set up networks of miniature recording devices in a forest, a field, or even on a cliff face. For example, the Kenya bioacoustics project led by Ciira wa Maina has built small cost-effective sound recorders based on Raspberry Pi miniature computers and deployed them to collect recordings of bird life in Kenya.

We can use such datasets to automatically detect and recognise birds, animals or insects, although in general it’s still a hard problem to detect every such sound. After all, the speech recognition software in your phone only has to deal with one species. The same machine learning methods can be used with animals, but they need to be adapted to the characteristics of the problem we’re trying to solve.

That motivated our work on automatically recognising bird species, and it also motivated our decision to turn our work into an app known as Warblr that anyone could use. You just record a bird with your smartphone and then Warblr tells you which species it thinks are most likely to be found in your recording, with a percentage weighting to show how sure it is for each species.

Can you tell the robin’s sound from the ring-necked parrot’s? Tomatito/Shutterstock

Sharing bird recordings

The app is not just a product. It’s also a “citizen science” project that allows us to ask people around the country to contribute the ten-second sound recordings they’ve made. These “sightings” can then be used for future research and bird monitoring. People contributed in their thousands, recordings such as this dunnock song:

But when we came to listen back to the sounds that the great British public had recorded since the app’s launch in August 2015 – more than 25,000 recordings from across the UK – we received quite a surprise. While 75% of the sounds we’ve analysed so far were made by birds, in a number of the recordings we couldn’t hear any birds at all. Instead, we found people had tried to imitate birds with their own voices, with varying degrees of success:

Bird or plane?

So now that we have a huge library of in-the-field bird recordings – and one of the best collections of bird impressions and miscellaneous other sounds – what are we going to do with it? In fact, even the non-bird recordings are useful for research because they teach the machines to distinguish all the different types of sound, from a screaming child to a screeching parrot.

We’ve learned something that has been corroborated by other researchers too. Even detecting whether or not there is a bird present (let alone what species it is) is still a tricky problem. That’s why we will be publishing the data we gathered and encouraging more research on detection and recognition of birds.

With machine learning, statistical analysis, and all these data sources, we will eventually have a much greater understanding of our bird populations, their migration patterns, their habitats and even their interactions with each other.

More of our favourite bird recordings are on the Warblr SoundCloud page:

You can help our research and listen to even more examples by taking part in our online listening game.

We hope that the recordings we’ve gathered will provide an invaluable insight into the behaviour of our British bird life, although it seems that we’ve captured an insight into the mindset of the British public too.