UNDERSTANDING RESEARCH: What do we actually mean by research and how does it help inform our understanding of things? Today a cautionary tale of why you should be careful of some new announcements made in the name of science.
It was dubbed a “spectacular” discovery – even “Nobel prize-worthy”.
But the March announcement via a press conference that researchers at the BICEP2 facility in Antarctica had detected the imprint of relic gravitational waves from a period of super-fast expansion in the early universe now looks a little shaky.
As an astrophysicist who works on the interstellar medium, I wouldn’t be surprised if the claimed detection eventually all went away. If so, this episode – much like the 2011 faster-than-light neutrinos story (more on that later) – is a telling example of the dangers of “science by press conference”.
The BICEP2 team invited the press to Harvard University to announce their result even before it had been refereed, alongside which they posted a now-infamous video of their champagne-on-doorstep announcement to Andrei Linde, one of the theorists whose work they claimed to have proved:
Peer review – where scientists evaluate the quality of other scientists’ work – may be the “least worst” system but it is the foundation on which modern science is built.
Open to review and scrutiny
Granted, the BICEP2 results were eventually published in a June edition of the journal Physical Review Letters – three months after the the initial announcement – and some would argue that putting it in the public spotlight involved being scrutinised by hundreds or thousands of expert opinions, instead of just one.
But the claim of discovery in science is a dangerous one, and in my view one best left to the gentle momentum built up over time as a result is confirmed by others and its importance understood.
At the very least it deserves waiting until the refereeing process has been completed, if only to avoid the embarrassment of the whole team should the result not hold up.
Of course, it’s not just in physics that the temptation of science by press conference or by press release exists – some notable recent examples include successfully cloning a human baby.
Another press release announcement that caught the media’s attention was that Oreos chocolate biscuits are as addictive as cocaine – but I’ll examine a couple of examples from my own field in detail.
BICEP2: flexed or flabby?
The issue with BICEP2 was a fairly simple one — to observe the cosmic microwave background (CMB) radiation, the relic radiation from the Big Bang that fills the universe at microwave wavelengths, you have to first subtract the glow of our own galaxy in the same wavelengths.
Take for example the recent maps (below) of the sky in microwaves from the European Space Agency’s Planck satellite.
The big red stripe is the microwave radiation from gas in the Milky Way, which has nothing to do with the microwave radiation from the early universe.
Subtracting this reveals (below) the small fluctuations in temperature from just after the Big Bang, which are the seeds of the galaxies that we see today.
A similar procedure applies to the observations of polarised light that was the basis of the claimed gravitational wave detection from BICEP2.
Polarised microwave light in our galaxy is produced by dust grains that spin on their axes and align like mini-compasses with the interstellar magnetic field.
Because the grains all line up in a similar direction this produces polarised light — precisely the kind of polarised light also produced by gravitational waves in the early universe.
Now comes the tricky bit
In subtracting the galaxy, the devil is in the details. The interstellar medium is a complicated place, full of turbulent, messy gas.
Understanding the foreground emission is therefore a complicated business, and one that requires very detailed maps of the galaxy itself (we have a saying in astronomy: “one person’s annoying foreground is another’s data”).
This is precisely what the Planck mission has been doing — so much so that the best results out of Planck for the first year or two were the exquisite maps they released of the gas and dust in the Milky Way, nothing to do with the cosmic microwave background.
By contrast, detailed maps of the dust polarisation from the Milky Way is precisely what the BICEP2 team were missing, so they had to rely on models of the dust emission to do the subtraction rather than actual maps.
So what? Well, it turns out they may have underestimated the foreground emission from dust, and therefore over-estimated the significance of any claimed gravitational wave detection.
Happily for science, we’ll be able to resolve this fairly soon: accurate maps of the polarised dust emission are exactly what the Planck team have been constructing, so they’ll be able to do a much better job of subtracting the foreground.
Indeed, Planck maps of the foreground dust emission, though not yet covering the patch of sky observed by BICEP2, are already being used to refine the foreground estimates, and these revised estimates are what is casting doubt on the claimed detection.
The need for quality control
Of course this is not the only case of scientists making their announcement too soon, before the research has been put forward for review.
Back in 2011 a team of physicists in Europe called a press conference to announce that they had measured neutrinos travelling faster than the speed of light.
The claim made headlines around the world because such as result would be in contravention of Einstein’s special theory of relativity that nothing could travel faster than light.
One prominent UK physicist and broadcaster, Jim Al-Khalili, even promised to eat his boxer shorts if the results were found to be true.
Well the claim was soon tested and debunked by other teams of physicists who could not repeat the results. That lead to some resignations from the original team that first made the claim.
The erroneous result was eventually blamed on some faulty wiring in the experiment’s equipment.
Let’s bring back humility in science
Rather than the endless chasing of headlines and press-conference-worthy claims of results I’d like to see more considered papers such as the original discovery paper for the cosmic microwave background, published in the Astrophysical Journal in 1965.
It landed astrophysicists Arno Penzias and Robert Wilson the 1978 Nobel Prize in Physics.
And what was the title of this ground breaking research? Just simply “A Measurement of Excess Antenna Temperature at 4,080Mc/s” and published in a regular journal. Possible explanations were left to others.
This article is part of a series on Understanding Research.
Further reading:
Why research beats anecdote in our search for knowledge
Clearing up confusion between correlation and causation
Where’s the proof in science? There is none
Positives in negative results: when finding ‘nothing’ means something
How to find the knowns and unknowns in any research
How myths and tabloids feed on anomalies in science
The 10 stuff-ups we all make when interpreting research