Opinion polls can have a profound influence on the atmosphere of a campaign. If they suggest that one side is well ahead, the media loses interest. The politicians get less space and perhaps airtime, while some voters decide that the result is a foregone conclusion and they might as well stay at home.
If on the other hand the polls suggest the result will be close, the media gets excited, the campaign gets more coverage, and voters are persuaded it might be worth their while turning out after all.
Until recently there seemed to be a real risk that the polls would dull the atmosphere of Scotland’s referendum campaign. Their results were proving to be remarkably stable.
Poll after poll last year was saying that the level of support for yes and no was largely unchanged. And with just one (much criticised) exception, all pointed to the no side being ahead.
November spawned a fluster
There was a danger that unless the no lead narrowed after the independence White Paper was published, Alex Salmond and his colleagues would find it increasingly difficult to keep their activists and raise money, while the rest of us would decide that the outcome was obvious.
But before the New Year was more than a few weeks old, it became clear that the no lead had indeed narrowed. Once the don’t knows were left to one side, the polls began to settle on an average yes vote tally of 42% rather than the 39% to which they were pointing before Christmas. Hardly a dramatic change, but enough to make a difference to the campaign atmosphere.
Indeed, by the end of last week it seemed as though the winning post might finally be well within the yes side’s sights. A poll conducted by Panelbase for the nationalist-leaning newsnetscotland.com put yes on 47% (once the don’t knows were excluded), just three points short of victory. The referendum race was now apparently well and truly on.
But in producing such a high yes figure, last week’s poll reminded us of another key feature of the referendum polls. They significantly disagree with each other.
Ouija board, ouija board…
Panelbase lies at one end of the spectrum. Even last year it was consistently putting the yes vote as high as 44-45%. It therefore needed no more than the kind of modest increase in independence support that most polls had already detected to push its estimate of the yes vote close to the 50% mark. To that extent last week’s poll result should have come as little surprise.
At the other end sits Ipsos MORI, whose most recent poll gave the yes side just 36% of the vote, and who has never put it above 38%. The remaining pollsters lie somewhere in between, with both Survation and ICM seemingly inclined to produce a somewhat higher yes vote than both TNS BMRB and YouGov.
Such substantial differences between pollsters are by no means the norm. A glance at all the polls of GB-wide voting intention conducted since the beginning of February, well over 50 in total, shows that not one has put the Conservatives on less than 28% or above 35%. The equivalent range for Labour support has been between 35% and 41%. These figures reflect little more than the variation from random ups and downs to which all polls are subject.
So the fact that Scotland’s referendum is not following this pattern shows that the polls have injected their own note of uncertainty into the campaign. They tell us the no side is ahead, but leave us still scratching our heads as to exactly how far.
Inevitably there has been quite a lot of debate about why these differences have arisen in the Scottish polls. There are some big differences in how the polls are being conducted.
For example, all three polls that tend to produce the highest levels of yes support – Panelbase, ICM and Survation – are conducted over the internet. Whether or not polls done that way are capable of securing adequately representative samples of voters remains a subject of substantial debate.
In contrast Ipsos MORI conducts its polls over the phone, while TNS BMRB still uses the most traditional method of all - going out and knocking on voters’ doors. And both are getting a lower Yes vote.
Yet these variations between the pollsters do not perfectly match the differences in how they obtain their interviews. YouGov’s more conservative readings (so far as the yes side is concerned) are also obtained via the internet.
In any event, most pollsters apart from Ipsos MORI attempt to overcome any risk that their samples are biased by adjusting their data so that the number of people they interview who say they voted Labour, SNP or whatever at the last Scottish Parliament election in 2011 reflects the actual outcome on that occasion.
Given that how someone voted in that election is quite a good guide to which way they say they will vote in the referendum, that should help ensure the polls are representative. One reason why this practice has proven insufficient to stop the pollsters from disagreeing with one another may be because half of Scots failed to vote in 2011, but may still vote in 2014.
Polling is in part an art of learning from experience. The lessons of the previous election can be carried forward into the methods adopted at the next one. But in the independence referendum, the pollsters are having to work out how best to measure something they have not previously had to estimate.
So they are all learning on the job. Consequently, perhaps considerable variation between them and continuing uncertainty about the likely outcome is something we are going to have to keep living with.
Of course, who is ahead and who behind is not the only important information imparted by polls. The proportion of voters who say they are undecided makes a difference to the atmosphere of a campaign too. Even if one side is ahead, the chances that things might change would seem to be much higher if lots of people say they are still undecided about which way they are going to vote.
A rush and a push and the land is ours…
Only last week SNP Westminster leader Angus Robertson could be heard attempting to lift the yes side’s spirits by claiming that as many as 40% of voters have yet to make up their mind.
At first glance, he would seem to have been overegging his pudding. With one exception, the polls agree that the proportion of don’t knows stands at around 15%.
That figure is not particularly high. Just two to three months before the GB-wide referendum on introducing the Alternative Vote in House of Commons elections, around twice that number said they did not know what they would do. Even six months out from the last UK general election some polls found as many as a quarter of voters in that category.
Yet TNS BMRB puts the proportion of undecided Scottish referendum voters much higher, at around 30%. That seems to be because since the autumn they have been asking people what they intend to do this September rather than how they would vote at the moment they are being asked. At the same time, Ipsos MORI has found that around a quarter of those who are willing to say which way they would vote now also say they might change their mind.
So although only around one in seven voters apparently have little or no idea at all how they will vote, at least as many if not rather more have an inclination in one direction or another but are not yet firmly fixed in their views. Between them they probably constitute around a third of the electorate, leaving Robertson’s estimate looking not so far off the mark after all.
But how might the SNP persuade these voters to change their minds? The answer seems straightforward – by persuading them of the economics of independence. According to ICM no less than 97% of those who think that independence would be good for Scotland’s economy are inclined to vote yes, while only 4% of those who think it would be bad are inclined to do so. The Yes side’s problem is that at the moment pessimists still outnumber optimists.
In contrast the argument that many in the SNP prefer to emphasise, and which Labour is spending much of its time disputing at its Scottish conference this weekend - that an independent Scotland would be a more equal society - matters much less to voters. Even among those who think independence would have that effect, less than two-thirds say they will vote yes.
But then given all the uncertainty, perhaps the politicians are simply being wise in sometimes ignoring what the polls are telling them?
John Curtice is running a website on the referendum polls at whatscotlandthinks.org