It’s that time of the year again. One of the biggest events in Europe’s (and the world’s) cultural calendar, the Eurovision song contest is legendary.
The attention paid to this bizarre show is enormous. And with such levels of investment and discussion, analysis and coverage, come the inevitable accusations of tactical voting and prejudice. This was notoriously seen in 2007 when Terry Wogan declaimed the “blatant bias” of the voting system. Many people get particularly upset about the supposed bias against the UK’s act.
The Eurovision song contest is decided by a combination of “experts” and tele-voting. People can phone in and score every act except the one of the country from which they are calling, with points between 0 and 12.
We decided to actually look at the available data and search for evidence of bias against certain countries – “negative bias” – in the scoring of other countries’ acts. We took into consideration contextual and act-specific factors, such as the sex of the performer and the language of the song. This was done to ensure that background differences (for example the fact that some countries may be more likely to enter female acts) didn’t induce further bias in the estimations.
Then we built a model to determine how far two acts sharing common features (like two female singers, singing in their own language in the same year of the contest) get different scores from different voters.
For each combination of voters and performers, we looked at the data over repeated instances of the last 20 years or so and based the bias on three main factors:
- A “geographical” effect, which accounts for potential bias due to spatial proximity
- A “cultural” effect, which we use to describe the underlying “clustering” of voters in a set of groups – we term this “cultural” in a rather broad sense
- A “migration” effect, which accounts for the fact that voters where a large population originally from the performer’s country may inflate the voting pattern in their favour
Our model does not seem to uncover any evidence of negative bias. There is no systematic propensity to give low scores to any given performer. But there is some (rather weak) evidence of positive bias. For example, we saw that some countries show a higher propensity to score some performers higher than others.
We rescaled these effects so that they are defined on a common range (between -3 to +4), in order to compare them and easily interpret the differences. This is the graph of propensity to vote for the UK:
In the graph, we present each voter’s average propensity to vote for the UK (the dots) together with a measure of uncertainty around this estimation. The darker intervals contain 50% of the entire probability distribution associated with our estimation, while the lighter ones include 95% of that distribution. So negative values can be considered as indicative of negative bias against of the UK, and vice versa. We consider that for an effect to be “substantial” the entire 95% interval should be below -2 (in case of negative) or above +2 (in case of positive values) – these thresholds are somewhat arbitrary, but there is some statistical theory that backs them up as reasonable.
Ireland and Malta are the only voters to have some positive propensity to score the UK highly. But these are not substantial and are not likely to make a difference. More interestingly, no voter seems to show a substantial negative propensity towards the UK acts – none of the intervals lie entirely below the -2 mark. And even more intriguingly, this finding is replicated for all the performers: no country is really subject to negative bias.
In fact, the model aligns with common sense, in terms of the mild evidence for some positive bias. This is seen in the clusters of countries that systematically tend to score a performer highly, over and above the “objective” characteristics of the act. But this is hardly a game-changer – the effects are never very large.
More importantly, we failed to find any evidence of “collusion”. In the past, Ukraine has shown a relatively marked propensity to score the Russian act high points (and who knows what will happen this year). But the reverse does not seem to hold – and again, this is a common feature to many of the cases we have analysed.
One of the interesting results from our model is that migration stocks may have an important effect in swinging the propensity to vote: for example, the Turkish acts seem to get high votes from the Germans – possibly as a result of the presence of a large number of Turkish migrants who can be bothered to first watch the Eurovision contest and then actively support their act by phoning in to score them highly.
Of course, this model cannot offer a definitive answer to this important and contentious question. And, we believe that other factors may be relevant, although difficult to measure objectively and so to include in formal modelling. For example, in the days prior to the final of the contest, one of the entries is usually suggested as the favourite. This may be based on objective qualities of the act, but it also hangs on political reasons, such as the willingness to take up the rather expensive enterprise of hosting the next edition of the contest. And what effect this may have on voting patterns is difficult to tell.
Hard Evidence is a series of articles in which academics use research evidence to tackle the trickiest public policy questions.