A recent announcement by the research director of the European Organisation for Nuclear Research (CERN) indicated the Higgs boson – often referred to as the “God particle” – has been excluded at 95% confidence.

That sounds convincing, sure, but what exactly does it mean?

Let’s say you order a coffee every morning. What if I told you that from now on you’d pay the same amount, but there’d be a 95% chance you’d get served your coffee, and a 5% chance you’d get served an empty cup. Sounds OK, doesn’t it?

Think about that, though – that’s a one in 20 chance you’ll pay for an empty cup … and you would expect this to happen once every 20 days on average, which is actually pretty frequent.

Standard deviations

When scientists talk about statistics such as those from CERN and the Large Hadron Collider (LHC), we’re really talking about standard deviations – a measure of how far any given data point is from the mean, or average.

It’s a little maths to calculate but essentially, if your data is “normally distributed” so that it is “bell-shaped” you would expect to see 68% of the data in the range between one standard deviation above and below the average.

The range of two standard deviations surrounding the mean would encompass 95% of the total data, which is usually a good estimate of “most of the data”. Higher and higher multiples of standard deviations cover more and more data.

This also tells you the chance of an event happening at random (as in not being a part of the normal distribution of data). We can turn our coffee example around and say that if getting an empty cup is a two standard deviation event, it could happen randomly 5% of the time.

When we say “we’re 95% confident” in something, we mean that if we repeated the measurement over and over, we would expect to see the same result 95% of the time. The problem is, as with our coffee example, 95% isn’t all that unlikely.

Five standard deviation events, with confidence levels around 99.99995% (a little less than a one in 2 million chance of data being outside this limit, or occurring randomly) are what physicists deem “interesting” discoveries.

But there are examples in which even these have disappeared when more data is analysed.

Watching a daily process, these events would happen once in 5,000 years.

These seem like minuscule odds, but if you play the lottery you take a much bigger risk. Correctly guessing six numbers out of a pool of 45 (in any order) has odds of one in 8,145,060.

Even if you played once daily, you’re only likely to win the top prize once every 22,300 years. Still, someone wins almost every week.

Is the Higgs hiding?

So what does this mean for the Higgs particle? To answer that, we need to know how likely it is to see one.

Rather than a daily process, the collisions at the Large Hadron Collider (specifically within the Compact Muon Solenoid experiment) occur every 25 nanoseconds (that’s 40,000,000 collisions per second).

These collisions happen between beams of trillions of protons, but on average only about 20 interact for each collision.

When the protons do collide, the interactions are actually between the component quarks and gluons.

When this happens, there are certain probabilities that further reactions will occur, such as two of the quarks each emitting a Z boson.

Then there’s a certain probability these will interact and annihilate to produce a Higgs boson.

If you add up the probabilities of each of these things happening to produce a Higgs, you get a very small number.

This, then, is the “total cross-section” for Higgs production – a measure of how likely we are to produce a Higgs.

Adding up all the ways that the Standard Model Higgs can interact, we get a very small, but non-zero total cross-section.

It gets better

Just to complicate matters, on top of the tiny probability of producing a Higgs, it’s much more likely that the collision would produce something completely normal with regards to the Standard Model that isn’t the Higgs.

We call this the “background” and it needs to be considered whenever we look for something new.

Now we’re looking for a small signal amid a noisy background of other everyday signals. Subtracting this “background” is a tedious task that must be done very precisely.

So it’s a needle in a haystack, but as I said, try to do something unlikely (but not impossible) enough times and it will happen.

After more than 18 months of data collection, the LHC finally has enough data to say something about the Higgs.

That data isn’t the complete picture though – we’re not sure how heavy the Higgs boson is (the Standard Model does not predict the Higgs mass).

Since matter and energy are related, we can’t tell what energy will be involved in interactions with Higgs bosons.

Different mass Higgs will produce different reactions, so we keep track of all these possibilities when we observe reactions in a mass/energy range.

This means we’re not only checking every interaction for signs that a Higgs was involved – we also need to check different mass/energy ranges.

Revisiting the cafe

Returning to our coffee example, imagine your usual cafe has this strange 95% policy, but you haven’t visited the other local cafes to ask what their policy is.

And imagine the size of the coffee cup changes between cafes.

The searches that have been completed so far at the LHC and the Tevatron at Fermilab have examined what some call the “low-hanging fruit” mass range – the energy range that, if the Higgs exists, would give the best signal-to-background-noise ratio and be the most obvious sign of a Higgs.

The range that remains to be checked is more problematic – getting good statistics in this region will take more time.

There’s also the possibility that the “Standard Model Higgs” is not the actual Higgs boson of the physical world.

Searches are also underway to detect or exclude various other Higgs models.

Back to the start

Let’s return to the original statement from CERN: “We’re 95% confident that the Higgs hasn’t been seen”.

This means: we’ve looked at the collisions enough times to say that there’s only a 5% chance that the Standard Model Higgs exists but we didn’t happen to detect it above random noise, after subtracting the background.

More precisely, if we performed the exact experiment again, say 2,000 times (that’s 2000 more 18-month data runs – we’re talking hypotheticals here) then if the Higgs exists, we would expect its signal to be buried under noise only 5% of the time.

We would then say either the Higgs doesn’t exist in this mass range, or this was one of those 100 runs which finds nothing.

If we increase our precision (by having more data, and thus fewer errors) we can say it’s less and less likely that this run happened to be one that found nothing by chance.

If we had another experimental search with a five standard deviation result, and we performed that experiment 2,000,000 more times, we would say either the Higgs doesn’t exist in that mass range, or this was the one run that didn’t see it. For practical purposes, at this level we place our bet that it simply doesn’t exist.

So what can we say about the existence of the Higgs with this “95% confidence” data?

We need to tread carefully. We can be 95% sure it’s not in the mass range excluded to that precision.

If we could rule out that region more precisely we’d have a couple of options:

1) If the Higgs does indeed exist, we’ve narrowed the possible masses it might have. This makes the odds of a detection in the remaining mass range higher, provided the Higgs does definitively exist.

2) If we factor in that we don’t know whether or not the Higgs even exists, our probability of finding it has actually decreased, since we’ve excluded a region it could be hiding in.

By exactly how much is hard to say, since the probability for the Higgs to have a certain mass is not even across the mass range.

But to say recent results “show” the Higgs doesn’t exist is factually wrong.

Of course, no amount of data can ever prove with 100.0% certainty the Higgs does not exist, and considerations of alternative theories are always useful to examine alongside mainstream theories. A five standard deviation is where we draw the line that something is believable, but even then we can be mistaken.

At present, the statistics on the remaining mass range do not provide any solid statement on the existence of the Higgs.

It’s not time to jump to conclusions yet, but given how complex the experiment and analysis of this matter are, it’s a major step forward towards a definitive answer.