UNDERSTANDING RESEARCH: What do we actually mean by research and how does it help inform our understanding of things? What if research throws up a result that calls for a new way of thinking? How do we handle that?
There are many misconceptions about science, including how science advances. One half-truth is that unexpected research findings produce crises, leading to new theories that overturn previous scientific knowledge.
Sometimes science progresses in this neat tidy fashion. But not very often. Assuming science is always so simple fuels misunderstanding of science, and provides ammunition to those who attack science, from cosmology to climate change.
Contrary to the myth, most anomalous findings have modest consequences. The vast majority of peculiar findings are usually the result of errors in data, methodology or misunderstanding the implications of existing theories.
Even when anomalies do prompt radical change, it is rare for them to completely upend large swathes of scientific knowledge.
Strange forces and Pioneers
In the 1970s, NASA’s Pioneer 10 and 11 spacecraft flew by Jupiter and Saturn before speeding towards interstellar space. As they coasted away from the sun, a strange “Pioneer Anomaly” was observed to be gently slowing the Pioneers. What was going on?
The Pioneer Anomaly has led to hundreds of papers, with many speculating on modified forms of gravity and relativity.
In principle the Pioneers could measure tiny accelerations, as they cruised through space. But they were never designed for precision tests of relativity, nor were they tested (prior to launch) to see if the spacecraft themselves produced tiny accelerations.
After decades of study, it appears the Pioneer anomaly had nothing to do with new physics. The Pioneers generate heat, and thus infrared light (photons), which were subtly pushing on the spacecraft (including via reflections). The Pioneer anomaly, rather than provoking a crisis and new physics, is a triumph of century old physics.
Other anomalies have appeared and disappeared in a similar fashion. But despite this history, media reporting of anomalous results often emphasises how the laws of physics could be overturned, rather than the likelihood of anomalous results disappearing. “Einstein Wrong!” works as click-bait for headlines, but is usually not true.
A personal tale of dark matters
I measure how galaxies grow, and at the end of the 20th century something seemed very wrong with galaxy growth research.
Simulations predicted the biggest galaxies should grow rapidly, as their vast gravity dragged in gas and neighbouring galaxies. In contrast, many observational studies found massive galaxies weren’t growing at all. What happened to all that gravity?
Some speculated that the dark matter paradigm was in trouble. Perhaps galaxies were less massive than people imagined. But instead of prompting radical change, this “crisis” has slowly faded away.
In 2007, I used a vast sample of distant galaxies to detect the slow growth of massive galaxies, and others have mitigated errors that have hampered observational studies of galaxy growth. Observational evidence for dark matter also improved, including cosmic microwave background measurements and the mass distribution within colliding clusters.
As computing power improved and simulations incorporated more complicated astrophysics, including supernovae and black-holes, the growth of simulated galaxies slowed down. So the gulf between simulation and observation closed.
The demise of this anomaly wasn’t as clean as that of the Pioneer anomaly. There were gradual improvements in both simulation and observation, and no single study tied up all the loose ends.
This gradual identification and resolution of anomalous results doesn’t always generate headlines, but it is often how science advances.
The scope for radical change
While most anomalous results fizzle and die, some do spark radical change.
The understanding of the world has been upended when scientific observations and theory have replaced pre-scientific ideas. For example, Galileo’s observations of planets resulted in heliocentric (sun-centred) models of the solar system replacing geocentric (Earth-centred) models.
Truly radical change can also happen when very limited data supports the previous hypothesis. Barry Marshall and Robin Warren won the 2005 Nobel Prize for Medicine for establishing that most stomach ulcers are caused by bacteria, not stress. While the stress causing ulcers had been widely accepted for decades, that hypothesis actually hadn’t been systematically tested.
As a science becomes more mature, with a wealth of supporting data, the implications of anomalous results become more limited. An example of this is Einstein’s general theory of relatively, which was (in part) motivated by odd measurements of the speed of light and the behaviour of Mercury’s orbit.
While general relatively has had huge implications for physics, it didn’t completely upend all previous physics. Maxwell’s equations for electromagnetism are still in use and Newtonian mechanics provides a good approximation of how satellites orbit the Earth.
The apple may have fallen on Newton’s head, but Einstein didn’t make the apple fly away.
Icy anomalies and the tabloids
While anomalous scientific results may seem a curiosity, they are central to public debates about science. To see why, go south!
Temperatures have increased over the past century as a result of increasing atmospheric CO2. The evidence includes (but is not limited to) lab measurements of CO2, measurements of atmospheric CO2, the spectrum of light radiated the Earth, planetary temperatures, and the pattern of temperature increase across the globe.
As a consequence, Arctic sea ice is decreasing, Antarctica and Greenland are losing land ice and sea levels are rising, and yet sea ice area around Antarctica has increased.
The increase in Antarctic sea ice area has been the subject of numerous articles by Andrew Bolt in the Herald-Sun and David Rose in the Daily Mail, among others. Some journalists believe this increase in sea ice is a fundamental flaw in global warming. But what can we conclude from this anomalous result?
The world isn’t getting any colder, so that doesn’t explain the increase in Antarctic sea ice. While simulations didn’t predict the increase in Antarctic sea ice area, they also didn’t predict the unexpectedly rapid decrease in Arctic sea ice either.
Sea ice area depends on air temperature, winds, ocean temperatures and currents, complicating the modelling of sea ice area. A simulation correctly modelling the greenhouse effect can fail to predict sea ice area if it doesn’t correctly model polar winds and oceans. While scientists are aware of this, such nuance is often absent from the tabloid media and blogsphere.
The tabloid media and blogsphere too often falls back on the simplicity of the myth, assuming the anomalous results will upend well-established science. This approach makes for good headlines and political point scoring, but the history tells us that science is very rarely upended in the manner some are wishing for.
This article is part of a series on Understanding Research.
Why research beats anecdote in our search for knowledge
Clearing up confusion between correlation and causation
Where’s the proof in science? There is none
Positives in negative results: when finding ‘nothing’ means something
The risks of blowing your own trumpet too soon on research
How to find the knowns and unknowns in any research
The 10 stuff-ups we all make when interpreting research