Well, at last we have a referendum poll that is generating some excitement. Hitherto every poll has put the No side ahead, and indeed often (as in Sunday’s YouGov poll for Devo Plus) quite a long way ahead. Many a commentator has been wondering whether the contest is already all over bar the shouting.
But now according to a Panelbase poll commissioned by the SNP and published today, the Yes side has a one point lead. Yes is given 44%, No 43%.
So how might we account for this seemingly surprising result? Can we safely conclude that there has been a sudden swing to the Yes side? The Panelbase/SNP poll is, after all, a little more recent than the YouGov/Devo Plus one – it was conducted between the August 23 and 28 (i.e. around the weekend before last) whereas the YouGov poll was in the field between August 19 and 22.
It is though difficult to think what has happened in the past fortnight that might suddenly have produced such a sudden transformation. In truth, the explanation is probably more prosaic.
Poll that likes to say Yes
First, as I have repeatedly pointed out, Panelbase’s polls (as hitherto conducted for Sunday Times Scotland) have consistently produced results that are more favourable to the Yes side than those of any other pollster. On average, previous Panelbase polls have put Yes on 36% and No on 45%. The average reading across everyone else’s polls is Yes 32%, No 53%.
So, on past form Panelbase have always looked like the company most likely to produce an unexpected Yes lead. Even so what it is still an apparent eight point increase in the Yes vote cannot simply be dismissed as most likely the result of “sampling error”.
There is, however, at least one crucial difference between the way that this Panelbase poll was conducted and the way the company have conducted their polls for The Sunday Times – and it is a difference that could well account for the apparent swing to the Yes side.
In Panelbase’s Sunday Times polls, how people propose to vote in the referendum has followed questions on how they would vote in a Scottish Parliament election. That was not the case in the SNP poll. Rather, it followed two questions that might well have helped cue some people into saying Yes that would not otherwise have done so.
First off, respondents were asked whether they agree or disagreed that, “Scotland could be a successful independent country”. No less than 52% agreed while only 37% disagreed. Then they were asked: “Who do you trust to make the best decisions for Scotland: the Scottish government or the Westminster government.” And as the Scottish Social Attitudes survey has repeatedly shown, people overwhelmingly say the Scottish government in response to a question like this, in this instance by 60% to 16%.
Push-me, pull-you
All survey researchers are aware that the responses they get do not only depend on the exact wording of the question they ask, but also on what questions have been asked immediately beforehand. There is good reason to believe this proved important in this case. By prefacing the referendum voting intention poll with two questions that elicited a response favourable to the Yes side, some respondents could well have been cued into saying Yes when they otherwise would not have done so.
The wording of the referendum voting intention question itself was a little different on the SNP poll too. Respondents were advised that there was to be a “referendum on an independent country”, whereas the Sunday Times polls have contained no such introduction. Those extra words might have helped the Yes side too. Conversely, at the other end of the spectrum, YouGov’s poll for Devo Plus described the referendum as a ballot on “Scotland leaving the United Kingdom and being an independent country” a description that might have been thought capable of discouraging some respondents from saying Yes.
It would be interesting to see what happens if in its future polls Panelbase do what every other pollster has in fact has been doing and that is asking referendum voting intention first in advance of any other questions on the subject at all. Given the continued popularity of the SNP when people are asked how they would vote in a Scottish Parliament election, perhaps having that question in front helps explain in part at least why Panelbase’s polls are always making the best reading for the Yes side.
So in truth this is not a poll we should get too excited about. There is good reason to believe the particularly favourable result for the Yes side is a consequence of the way in which the poll was conducted. That, however, is not to say it should be ignored. Evidently, if the Yes side could persuade people that independence would deliver a successful country free of unwanted Westminster rule, then they might well drive people into their camp. But we should not as yet presume that they have suddenly made significant progress in actually achieving that task.
This article originally appeared on whatscotlandthinks.org. Click here for full details of all Scottish referendum vote intention polls to date.