Polls, framings and public understandings: climate change and opinion polls

Can respondents make an informed response in an unexpected telephone poll? Esparta/Flickr

I, along with many Australians, listened to the news coverage on Monday morning of the Lowy Institute’s annual survey, with reasonable disappointment and initial surprise.

This is a respected polling exercise. The institute’s mission and mandate has been to take the nation’s pulse with respect to selected public opinion and foreign policy issues once a year.

Climate change was one such issue in the telephone survey of 1002 individuals undertaken and in part interpreted by a market research company between 30 March and 14 April of this year.

The published report suggests 29 questions were asked with three of these questions directly related to climate change.

The executive summary states that “Support for taking tough action continues to erode. The foreign policy goal of tackling climate change is considered very important by only 46% of Australians … Support for the most aggressive form of action to address global warming slipped five points from last year, with 41% saying global warming is a serious and pressing problem and that we should begin taking steps now even if this involves significant costs.” These summarised findings were then duly covered by the media.

How do the Lowy Institute findings compare to other studies?

These “survey” findings, at first glance, would seem to be inconsistent with the findings of many Australian and international surveys over the past several years. These surveys include those by CSIRO, Ipsos-Eureka, Cardiff University, Stanford University, and Yale University.

They are certainly at seeming odds with our own national survey findings from 12 months ago.

In our own representative national survey 3096 Australians spent between 30 minutes and an hour completing an on line survey which, unlike an unexpected telephone call, allowed them to carefully read and think about their responses. These respondents were given a convergent set of 140 plus questions relating not only to the nature and threat of climate change, but to Australia’s energy future, extreme weather events, perceived environmental changes, their direct experience and connection with the natural environment, and their responses to these indirect and direct encounters.

Our findings, available on the NCCARF website, clearly indicate that our respondents were, overall, very concerned about climate change. They felt that they themselves were responsible for taking action against climate change. They very definitely felt that national governments and the international community were responsible for and should be taking action (77% and 71% of respondents, respectively).

Our research was undertaken in tandem with a similar national survey in Britain by the Understanding Risk Centre of Cardiff University. Findings about public risk perceptions, beliefs, concerns, and energy options were in most instances strikingly similar.

And these findings are similar not only to other substantive Australian survey findings, but to highly respected survey research findings over the past few years by research teams at Stanford University, Yale University, and other academic research institutions in North America and Europe.

The challenge of documenting public perceptions and views

So what is happening here? We need to firstly appreciate that “climate change” is a highly charged “political issue” in Australia, as well as being an emotive “social issue” and “environmental issue”.

Reference to “climate change” can include everything from global atmospheric systems and greenhouse gas concentrations to the threat and projected consequences of climate change for ecosystems, human communities, and political and economic systems.

Some would argue that “climate change” has become not only cause, consequence, and account, but an encompassing meta-narrative and moral tale of what seems to be happening to the world’s environment, its life-support systems, and contemporary life in a vastly changed and changing world. Hence reference to “climate change” in a survey question can evoke many things.

But climate change is also the most well-researched phenomenon, threat and challenge to human communities that the world has encountered. So it is not so surprising that such a multi-faceted and consequential phenomenon is the subject of considerable consternation as well as contested sense-making, often on the run.

It is also not surprising that single item and variously framed complex questions, embedded in telephone surveys, addressing political and policy opinions are somewhat unstable barometers, providing intermittently different pictures and interpretations.

What did the Lowy Institue ask?

Let’s look at the principal climate change “tracking question” being used in this Lowy Poll.

“There is a controversy over what countries of the world, including Australia, should do about the problem of global warming. I’m going to read you three statements. Please tell me which statement comes closest to your point of view.

  • Global warming is a serious and pressing problem. We should begin to take steps now even if this involves significant costs.

  • The problem of global warming should be addressed, but its effects will be gradual, so we can deal with the problem gradually, by taking steps that are low in cost.

  • Until we are sure that global warming is really a problem, we should not take steps that would have economic costs.

  • Don’t know/Refused

(italics are those of the present author)

“Dealing with global warming” The Lowy Institute Poll 2011

How many would be able to keep these three options clearly in their head when responding? Who could provide a reflective response in the context of being asked 29 other such national and international policy questions over the phone? What if one did not particularly endorse any of the three options in this case?

And the initial question and response options use the North American term for climate change, flag controversy, uncertainty, and costs - and the international context and implicit matter of whose responsibility.

There is also the use of language such as "until we are sure” and “so we can deal with the problem” in the second two options.

The question and response options are arguably more than “double barrelled”; they contain multiple and emotional button-pressing matters and language.

The other two Lowy Poll climate change questions relate to the government’s efforts in addressing climate change and willingness to pay. This reflects an understandable policy and economic cost review rather than a researching and reporting of public understandings and responses to a salient issue and concern.

It is a common enough practice in such political polls to address multiple matters in single item survey questions such as this. But such questions and response formats do not lend themselves to careful measurement and monitoring of public risk perceptions, understandings or responses to a complex issue such as climate change. Nor do they lend themselves to measured responses on the part of those reporting and interpreting the already interpreted findings in the executive summary.

But let’s not dismiss the fact that 41% of Lowy Poll respondents selected the response option stating that “Global warming is a serious and pressing problem. We should begin to take steps now even if this involves significant costs.”

It is also clear in our own data and that of others that Australians are concerned but conflicted about “climate change”. They have real concerns about the cost of living, financial hardship, and equity issues, ironically but quite understandably in the wake of multiple recent, convergent, and devastating natural disasters. Hence survey questions can be rather, albeit unintentionally, loaded.

Is there a better way?

In Australia, social scientists, supported by universities and research organisations such as CSIRO, have recently begun systematic, longitudinal social science research programs addressing public climate change perceptions and understandings. In these organisations, other and complementary climate change science research programs are integral components of strategic and interdisciplinary research fronts focussing on climate change adaptation and mitigation.

These in-depth research and monitoring programs are examining important changes and impacts taking place in the human landscape in association with global and national physical environmental changes. They provide a far more sensitive and informative avenue for benchmarking public understandings and responses to the threat of climate change, including associated social and political issues and concerns, than do headline findings from political polls serving very different needs and audiences.

The reality is that in our information-saturated, sound-bite world, we need to examine media reporting – and interpretation - of “survey” findings with an informed and critical eye.

And if we are serious about measuring, monitoring and understanding important changes taking place in the human landscape we need more than the “snapshots in time” based on the several single item tracking indicators that many polls provide.

So, yes, these particular survey findings and their interpretation are different from our own findings and interpretations of our respondents’ climate change risk perceptions, understandings, and responses.

But there now exists a more stable and coherent emerging picture of public understandings of climate change around the world, based on convergent and substantive social science research findings, which resonates strongly with our own Australian findings.