At the heart of the scientific method is the process of hypothesis testing. Given an observable phenomenon in the world, a scientist will construct a hypothesis which seeks to explain that phenomenon.
Hypothesis testing is used by pharmaceutical companies to ascertain whether a drug is effective against a certain disease, by neuroscientists to determine whether neuroplasticity-based therapy helps stroke patients, by advertising businesses to decide whether a new campaign is worthwhile, and so on.
The way hypothesis testing works is by setting up two opposing hypotheses. One, the “null hypothesis”, is the reference or baseline hypothesis.
If the null hypothesis is supported, nothing unusual is going on; the factor under investigation has no explanatory power; the drug being tested has no effect; the advertising campaign doesn’t work.
But don’t be misled – this hypothesis is crucial. In reality it is the only hypothesis actually being tested.
The other side of the coin is the alternative hypothesis: the interesting and challenging contender, the hypothesis that may lead to new discoveries, decisions and advances. The drug that’s been tested does work; the advertising campaign is a smash hit.
Consider the fictional characters Dr Nool and Dr Altman, two researchers collaborating in the study of sleep disorders. One night, Dr Altman falls asleep on his couch after eating an apple. The following morning he knocks on Dr Nool’s door and asks: “What if apples could ease insomnia?”
To turn this simple question into something worth investigating scientifically, the doctors come up with the following hypotheses:
- Null Hypothesis: eating apples does not improve sleep quality
- Alternative Hypothesis: eating apples does improve sleep quality
A simple test would consist of splitting sleep disorder patients randomly into two groups. Patients in Group A (for “apple”) could be asked to eat an apple after dinner for a given period of time, while patients in Group C (for “control”) would be asked to eat a piece of fruit other than an apple.
Note the importance of being specific with the hypotheses. The question was whether apples, not fruit in general, could be used to tackle sleep disorders. Consequently it is convenient to ensure the two groups only differ in the factor under study (the effects of eating an apple).
Hypothesis testing is essentially a statistical procedure that calculates probabilities. It is easy to see that a crucial aspect of any test is to use a large sample size.
Imagine that in our apple enquiry, there are only four patients in each group. Three patients in Group A (75%), compared to two in Group C (50%), report improvements to their insomnia after eating fruit.
Can we conclude that eating apples works to ease insomnia? Not really.
Chance alone can explain those results. But if the trial was run with 4,000 patients in each group and found the same proportions we would be much more confident in the “apple effect”.
(As an aside, next time you see a statement such as “nine out of ten doctors recommend …” refer to the small print and ask yourself how these doctors were sampled.)
Objectivity and scientific bias
Hypotheses developers and testers usually hope that the null hypothesis is rejected and their alternative hypothesis supported – that the drug they’re testing is effective; the campaign they’re running is a success; that the light is bent by gravity as predicted by Newtonian physics and Einstein’s theory of relativity …
But this desire should have no bearing on the test. The design of the experiment and the subsequent data analyses should be completely objective and unbiased.
The test must be as conservative as possible because the stakes are high: understanding nature, improving people’s health and safety, the future of a company or a new technology.
It might seem obvious, but until proven otherwise, the null hypothesis is true. As one classic textbook on statistics nicely puts it:
As in a jury trial, the burden of proof rests with the alternative hypothesis; innocent until proven guilty … When you test a hypothesis, you must act as judge and jury, but you are not the prosecutor.
Hungry for more?
So here’s a more immediate example of an alternative hypothesis: “reading this article will get you closer to the ‘two-fruit-and-five-vegies’ rule today”. C’mon, admit it: after so many references to apples and Newton you must be nibbling on some fruit by now, or at least craving it.
The accompanying null hypothesis would be something along the lines of: “reading this article has no effects on your cravings for fruit”.
And so you, dear reader, get the final say. Should we reject the null hypothesis above? You can post a comment with your reply. Once I have a large enough sample size I will look at the data.
Until then, I’ll withold judgement.