Citizen science occurs when data for scientific research is collected by members of the public in a voluntary capacity. Public participation in environmental projects, in particular, has been described as a global phenomenon.
But there is a stigma associated with these types of projects. The data collected are often labelled untrustworthy and biased. Research in this area continues to show however, that data collected by what is essentially a non-professional workforce, are comparable to those collected by professional scientists.
Provided steps are in place to deal with data integrity, we have much to gain by putting more trust in citizen scientists.
Across the globe thousands of people collect data on everything from counts of stars in distant galaxies to the timing of flowering events. Volunteers have long been collecting data on the health of coral reefs, and ornithologists encourage volunteers to collect data on bird migration.
Citizen science has benefits for scientists – including an inexpensive and potentially large labour force – and citizens, who get knowledge and fulfilment. These schemes expose people to the environment and develop the stewardship ethic.
But what motivates my interest in this area is the potential to create a more scientifically literate society; building the capacity for people to take information they receive in their everyday lives and then being able to make informed choices based on the what they have learned. Those choices could be anything from the products they buy as consumers or the political parties they support.
Sounds good: what’s the problem?
While citizen science projects vary in their study subjects, the ecosystem of interest and objectives of the research, they all face one important challenge that jeopardises their success, sustainability, integrity and in general, acceptance in the wider scientific community. Many scientists question the quality, reliability and in general, the utility of data.
Evidence for the stigma associated with citizen science projects comes from different sources. One was a long standing community-based program to survey diversity of bird species. Researchers found the estimated numbers of birds changed through time simultaneously with changes in the observers. It was concluded that the trends detected were not likely to represent real changes in bird abundance, but were more likely due to prejudices of the individual observers.
The negative perception of citizen projects is not new. Twenty years ago, the use of volunteer data came into the international spotlight when an amendment was made to prohibit the US National Biological Survey from accepting the work of volunteers. This was supported by two arguments in the House of Representatives declaring that volunteers are incompetent and biased.
Are volunteers' data that bad?
Questions over data integrity continue to this day. It’s surprising, because a growing body of literature shows that data collected by citizens are comparable to those of professional scientists.
For example, researchers have detected no differences between field samples of aquatic invertebrates that were collected and identified by volunteers and professionals. A similar study showed that data collected by volunteers and scientists agreed 96% of the time.
Both studies concluded that volunteers could collect reliable data and make assessments that were comparable to those made by highly trained professionals.
My own research on vegetation metrics collected as part of ecosystem restoration projects also showed that the degree of agreement of data collected by volunteers can be as good as those recorded by professional scientists.
Results showed that scientists as a collective group collected data that was in closer agreement with “the truth” than those of volunteers. But when data collected by individuals were analysed, some volunteers collected data that were in similar or closer agreement to the truth, than scientists. Both groups’ estimates were in closer agreement for particular attributes than others, also suggesting that some attributes are more difficult to estimate or are more subjective than others.
An important message from these studies is that data-integrity issues can occur. But it’s just a matter of honing in on those particular issues and addressing them if necessary. This can be through training to improve skill sets or calibrating data where possible.
It should not be a case of blaming the citizens. The scientist behind such programs should have checks in place – citizen science project or otherwise!
A final thought
It is perhaps too simple to reason that the “success” or “failure” of a citizen science project will be determined by the degree to which data collected by volunteers agree with those of scientists. Citizens in environmental projects using self-funded volunteer labour have, and should be valued for more than just the potential for cost savings.
Research has shown that volunteers experience high levels of many different satisfactions. For example, participants gain educational benefits, there is potential to reunite science and society, the public can be inspired to appreciate nature through hands-on experiences and participants can build a sense of ownership in both the program and place.
These unmeasured benefits are achievements in themselves and it may be far better to concentrate on these as a focus of success. Such benefits will engender a society that will protect our environment at a time when it’s needed most.
Want to get involved in citizen science? The Atlas of Living Australia is a good place to start.
Mike Coleman
Land Use Planner
This kind of scepticism of citizen science is not something that had occurred to me. The South African Bird Atlas Project2 has been running since 2007 with about 1000 observers. Based in the Animal Demographic Unit at the University of Cape Town. Most of the potential problems mentioned here have been covered in the project design. It gives contributors almost real-time feedback of their records on the website, which itself has a wealth of data in pentad map form for species, distribution, season, location.
Try the website and links at http://sabap2.adu.org.za
James Jenkin
EFL Teacher Trainer
Is there a greater risk of bias when someone with a passionate interest in a a cause collects data?
Les McNamara
Researcher
Perhaps. Paid and unpaid data collectors can have a passionate interest in their subject matter. There must also be risks if data collectors are disinterested in the subject of the research.
Suzy Gneist
logged in via Facebook
The same could be asked for scientists who may benefit (esteem or money) from certain results - I would have thought there was less risk to be biased where volunteers with no financial/professional stake are involved.
James Jenkin
EFL Teacher Trainer
Yes, fair points, Les and Suzy. I guess citizen researchers may be less aware of research protocols - but you'd assume any project methodology would address this?
John Gollan
Research Fellow, School of the Environment at University of Technology, Sydney
James - in response to your comment 'I guess citizen researchers may be less aware of research protocols', my experience says they are. I think making them aware of bias should be part of any 'training' that citizens receive. If they know the 'pitfalls' then this should, in theory at least, inform them of the consequences. One of the consequences includes the potential that their data/project (which is hard work to collect!) is rendered useless by the 'snobby' scientists (in references to other comments).
Objectivity should be an important part of training in citizen science.
Les McNamara
Researcher
It is odd that a society can train volunteers to reliably perform first aid, fight fires and provide care to needy and vulnerable people, but that same society can't trust volunteers to count birds. Scientific snobbery?
John Gollan
Research Fellow, School of the Environment at University of Technology, Sydney
A good point Les. I also find it odd that postgrad students, postdocs and tenured academics in lots of different fields (especially in mine of ecology) often enlist their friends, family and significant others to help them with fieldwork and data collection. Strictly speaking, they are no different to citizens in many citizen science projects.
Henry Midshipman
logged in via Twitter
There certainly is a role for Citizen Science. I was checking out the latest “definitive” Australian ACORN-SAT climate data at http://www.bom.gov.au/climate/change/hqsites/. I thought I would pull out a couple of daily minimum temps over the past few years for a few stations - drop them into a spread-sheet and have a look at the numbers.
Last year 2012 was a leap year with 366 days and a February 29 - but not according to the BOM - there was no 20120229 data.
This omission must cast doubt…
Read moreGrendelus Malleolus
Senior Nerd
Which data set was this? Mine all show the 29th.
Stephen Prowse
CEO at Wound CRC
I think there may well be a degree of scientific snobbery involved with tension in the scientific community between scientists who recognise the urgent need to expand the range of ecological data collected and those who believe that only trained scientists can reliably collect ecological data.
With declining resources and an increasing need to monitor changes to our environment, "citizen science" should be supported more strongly than ever. With an increasing number of active seniors, there is a well educated, highly committed group of observers who can readily collect reliable data that has the potential to be of great value in monitoring rapidly changing ecosystems.
Dianna Arthur
Dianna Arthur is a Friend of The Conversation.
Environmentalist
Stole my thunder, Stephen.
There is a great deal of snobbery between the sciences (eg biology v physics) let alone the divide between public understanding and the work of scientists.
To deride or ignore the contribution made by lay people to scientific knowledge is a foolish stance to take.
More articles promoting science with the average person's experience please!
Tim Scanlon
Debunker
I wouldn't call it snobbery Stephen. A biased dataset is hard to draw any meaning from. So it is important that this bias is taken into account somehow. Unfortunately, many projects lack the statistical and/or scientific support to make this possible.
John Gollan
Research Fellow, School of the Environment at University of Technology, Sydney
Stephen - I completely agree. In an environmental context, I think citizen science has a potential to engender a society that protects the environment in the first place. It's not until you get out there and see for yourself, that you start to see the damage we have, and can do.
Judit Szabo
Research Fellow, Threatened Bird Conservation at Charles Darwin University
Thanks for raising awareness for the value of volunteer-collected data John.
Our results comparing a "weakly structured mass-participation bird atlas survey" and a "robustly designed monitoring scheme" have the same conclusions (http://onlinelibrary.wiley.com/doi/10.1111/j.1474-919X.2012.01229.x/full). We conclude that appropriate calibration using selected regional surveys, including surveys to estimate absolute densities, can enable volunteer-collected and weakly structured atlas data to be used to generate robust occupancy and minimum population estimates for many species at a regional scale.
also issue #64 of Decision Point (October 2012) http://decision-point.com.au/past-issues/dpoint64.html details the value of volunteer-collected data through different case studies.
John Gollan
Research Fellow, School of the Environment at University of Technology, Sydney
Thanks Judit - and thanks for posting the link. I had read this one - one Id recommend as well.
Gary Goland
Gary Goland is a Friend of The Conversation.
Researcher
The potential of citizen science and reliability of data presented has been strengthened as the technical elements in collecting data have changed. This comment relates particularly to photographic observational information that is a significant part of information submitted to the Atlas of Living Australia. The certainty of the data contained in the apps documentation that is utilized by digital cameras with GPS facility, strengthens all survey observations. The data it generates introduces opportunity…
Read moreJim Wright
Retired Civil/Structural Engineer, IT Consultant/Contractor
I run a blog (reengineeringaustralia.com, if anyone is interested) on sustainable population, food production and the like, based on the thesis that, if we don't utilise the resources in our island nation to the max, other hard-pressed countries might try and do it. I have proposed a one-kilometre square grid over the entire island and adjacent sea waters, identified by GPS co-ordinates and with the initial data collected from satellite imagery. The resulting database would be public (three cheers…
Read moreTrevor Hausler
Waterwatch Coordinator (Merri and Moonee Ponds Creeks)
I am interested in your statement that researchers have detected no difference in aquatic invertebrates collect and identified by volunteers as opposed the professionals. As a Waterwatch coordinaor one of the issues we grapple with is the accuracy of the information collected, particularly identification of macroinvertebrate. I would like to read any research on his topic.
John Gollan
Research Fellow, School of the Environment at University of Technology, Sydney
Trevor - just a few to get you started...
Boudreau SA, Yan ND (2004) Auditing the accuracy of a volunteer based surveillance program for an aquatic invader Bythotrephes. Environmental Monitoring and Assessment 91:17–26
Boylen CW, Howe EA, Bartkowski JS, Eichler LW (2004) Augmentation of a long-term monitoring program for Lake George, NY by citizen volunteers. Lake and Reservoir Management 20(2):121–129
Delaney DG, Sperling CD, Adams CS, Leung B (2008) Marine invasive species: validation of citizen science and implications Environmental Management (2012) 50:969–978
Engel SR, Voshell JR (2002) Volunteer biological monitoring: can it accurately assess the ecological condition of streams? American Entomologist 48:164–177
Fore LS, Paulsen K, O’Laughlin K (2001) Assessing the performance of volunteers in monitoring streams. Freshwater Biology 46: 109–123
Dave Slutzkin
logged in via Twitter
Does this "citizen science" include health sciences? People implicitly "collect" data about their own health every day, especially from the point of view of diet and exercise and their relationship to their general well-being. These things are notoriously difficult to study - but "crowdsourcing" the data collection could make for an interesting amount of power.
Tim Scanlon
Debunker
I currently work with some groups that do their own research. While there are huge benefits in terms of demonstrating concepts, groundtruthing for local conditions, etc, it is really hard to do original research.
The main reason is replication and variability. Even among the science community it is hard to get adequate coverage of replication and randomisation to get good data. Try doing this with the average person and shortcuts are invariably taken. As such, citizen science needs to have support…
Read moreGary Goland
Gary Goland is a Friend of The Conversation.
Researcher
Hi all interested in the topic. Tim it is good to hear of your positive involvement with projects that utilize volunteers giving time, and their willingness to engage what can be useful data. With many years experience in medical physiology experimental work, I relate the detail of variability of data, and how much that underlines the acceptability of the information derived. Much of this can be traced to experimental design, but at time, also genetic variability within the experimental population…
Read moreTim Scanlon
Debunker
Gary, as long as enough data is collected at sampling time, then I agree.
Observation is the key to science, but it has to be replicable and analytical, otherwise it is hard to tell if the observation was correct or not. For example, my wife found that a weed species was being confused for a native species. This was a mistake that many had been making for a long time because the correct observations were not being made.
Gary Goland
Gary Goland is a Friend of The Conversation.
Researcher
Hi again Tim. Perhaps suggest those unfamiliar with the species they are observing, consider contact with Local Council biodiversity officers, or the museum staff. For plants particularly, the herbarium. The Atlas of Living Australia is partly staffed by people in museums in most states, so they are already receiving and greatly assisting species identity for members of the community. Learning from mistakes is part of science itself. Correction is part of the shared education process that will again add greater value to the long-term collaborative activity of citizen science.
Tim Scanlon
Debunker
I was more pointing out that the expertise of those involved may not be adequate enough for accurate observation. Thus, it is best to have citizen science collaboration with actual scientists.
John Gollan
Research Fellow, School of the Environment at University of Technology, Sydney
Tim - some good points and I agree entirely. I think support from scientists is needed at all times. It is unrealistic to expect a community group to instigate, design, implement and report a rigorous research project. Citizen scientists should never be dumped with ideas and tools - many examples I have seen in my experience!
Lawrie Conole
Consultant ecologist & PhD candidate
I'm always fascinated and amused in equal parts when the subject of citizen science is raised. I should 'fess up early that I started out in my teens as a citizen scientist (birder/naturalist) and crossed over into the scientific professional ecologist realm at some point during my career. I'm now bookending all of that with an ecology PhD which I'll finish at the age of 52.
Those who are least familiar with the subject organisms of research are as often postgraduate researchers as amateur volunteers. Whilst experimental design might be the near exclusive province of professionals, familiarity with the natural history and ID quirks of flora and fauna may be stronger in the ameteur community in many cases.
As someone who has a foot in each camp, so to speak, I'm all too familiar both with scientific chauvinism towards amateurs and amateur suspicion of 'boffins', and always keen to find opportunities to bring the two ends of the spectrum together so they can learn from each other.
Nev Norton
Farmer
I would be quite concerned about the accuracy of the data.
In some fields of research particularly environmental, volunteers may have a vested interest in skewing a particular outcome, perhaps to support an ideological or political outcome. After all it's the people who have the vested interests who will have the highest motivation to volunteer to do the work, how do you guarantee impartiality and non manipulation or falsification of data.
Peter Hindrup
consultant
It has long puzzled me that non academics are so often discounted. Farmers, many of them know when plants flower. That is they know something is flowering a few days early, or late, and often what significance this has.
My father and younger brother could recognise midst the myriad overlaid prints on a path the prints of a pig, deer, cattle beast, and tell you how far ahead it/they were. Me, on a clean piece of ground, one clear print, I too could probably tell you what animal it was. They did…
Read moreClare Hawkins
Zoologist
In a way it's a funny conversation to be having: 'Can people who haven't been trained help with a study?". Of course effective contribution by anyone to any study will require some sort of training in how to make that contribution. This could take minutes or years, depending on what is required. Key to the study design is ensuring that the requisite training is effective, that all resources including manpower are used appropriately, and that potential methodological errors including observer error…
Read moreJohn Gooderham
Freshwater Ecologist at The Waterbug Company
I agree that a lot off this discussion hangs off study design. But "Excellent" study design is a bit vague. Specifically, the design has to incorporate an understanding of the types of errors involved in the data collection and understand that these errors will be different for volunteers when compared to the errors in data collected by trained scientists (yes they do exist).
Read moreWe recently started a program with DSE in Victoria where we changed the taxonomic level that was used for data collection…
Bailey Erin Smith
logged in via Facebook
John,
Do you have any sources you could link me to regarding how the House and Senate have treated citizen science? I was intrigued by your reference to the US National Biological Survey. Has this happened with any of the regulatory agencies? For instance, the EPA actively uses citizen source water quality data.
John Gollan
Research Fellow, School of the Environment at University of Technology, Sydney
Hi Bailey, this was a reference to a statement made in a paper by Chris Newman et al (see below). You may be able to find what you need there.
Newman C, Buesching CD, Macdonald W (2003) Validating mammal monitoring methods and assessing the performance of volunteers in wildlife conservation–’’Sed quis custodiet ipsos custodies?’’. Biological Conservation 113:189–197
John Rodger
Director Wildlife Biodiversity CRC Bid
Thanks John for stimulating such an interesting and comprehensive conversation. As Steve Prowse and others have noted there is a good bit of professional snobbery behind the low opinion held by many of citizen science. For me involving the broader 'non-expert' community in meeting and dealing with the major environmental challenges we face is not an option its essential. The challenge the scientific community faces is to build on the research and experience of those involved in citizen science like John Gollan to optimise its effectiveness.
Gary Goland
Gary Goland is a Friend of The Conversation.
Researcher
To offer reference to a further project utilizing community participation, undertaken very recently by Prof Chris Daniels etal, I point to The Great Koala Count, http://www.unisa.edu.au/research/barbara-hardy-institute/research-projects/the-great-koala-count/ . Chris and others involved the project disclosed great enthusiasm and benefit to all parties. It encouraged utilizing GPS and an APP to relate detail. It utilized the Atlas of Living Australia facility. A wonderful example of the extraordinary information that can be gathered in a short space of time by the community. The ABC radio also strongly supported this and other community projects Chris has run in the last 10 years. It can work
Jonathan Stone
logged in via Twitter
John, great article. You mention that Citizen Science could reunite science and society - do you have any links to any articles/research supporting this? I don't disagree...I am just looking for anyone that has tested this...
Cheers.
John Gollan
Research Fellow, School of the Environment at University of Technology, Sydney
None that i can think of without digging through my literature, but one article that makes a good argument is Devictor et al 2010. You should find some good refs in their list.
http://onlinelibrary.wiley.com/doi/10.1111/j.1472-4642.2009.00615.x/abstract
Paul Lindgreen
Administrator
Scientific methodologies now exist for citizen science based research to validate the accuracy of data. Yes, there are many citizen science projects that do not use these methods....but they were not intended for peer reviewed journals. Over 100 peer reviewed research articles based on citizen science exist, the stigma is gone, it's legetimacy has been proved beyond doubt, to deny its value in the presence of such evidence takes great ignorance and stubborness.
Using these well documented methodologies researchers and other project coordinators can even setup and manage their own local citizen science project with free online tools at WildlifeSightings.net , now everyone can truly participate in scientific research, no one is restricted by requiring big research grants.