The mass testing of asymptomatic people for COVID-19 in the UK was thrown into question by a recent study. In a pilot in Liverpool, over half the cases weren’t picked up, leading some to question whether using tests that perform poorly is the best use of resources.
The tests involved in this study were antigen tests. These see whether someone is infected with SARS-CoV-2 by identifying structures on the outside of the virus, known as antigens, using antibodies. If the coronavirus is present in a sample, the antibodies in the test bind with the virus’s antigens and highlight an infection.
Antigen tests are cheap and provide results quickly. However, they are not always accurate. But what do we mean when we say that a test is inaccurate? And is it really the case that “an unreliable test is worse than no test”?
Sensitivity vs specificity
When testing, one thing we’re interested in is how good a test is at detecting the virus in people who are actually infected. The more sensitive a test is, the less likely it is to deliver a false negative result to someone who has the virus.
False negatives can have significant costs. If people receiving them are also infectious, this may increase the risk of viral transmission, as they’ll behave as if they don’t have the virus – what’s known as “false reassurance”.
But sensitivity is not the only kind of accuracy that matters – we’re also interested in how good the test is at providing positive results only to those who are actually infected. The more specific a test is, the less likely it is to deliver false positives to those without the virus. False positives also have costs – a person’s liberty might be restricted even though they pose no risk of transmission.
The Liverpool data
Preliminary data from the Liverpool pilot suggests that the test used was 48.89% sensitive. That translates into a very high false negative rate, risking widespread false reassurance. The test cannot robustly confirm that someone isn’t infected.
However, there are other relevant points to consider from the Liverpool pilot. First, the study found that the specificity of the test was 99.93%. That means that only a small proportion of participants who weren’t infected were given a positive result by the test. This specificity is a good thing, but we shouldn’t overstate its importance; high specificity alone does not entail that a positive result is likely to be a true positive. This likelihood, or the test’s “positive predictive value”, is also partly determined by how prevalent the virus is in the tested population.
For instance, say you test 100,000 people with a test that is 99.93% specific, yet the rate of COVID-19 in this group is relatively low – only 70 cases per 100,000 people. Among the 99,930 people who are uninfected, the test would still return a false positive result to 0.07% of them – roughly 70 people. So in this scenario, assuming the test is perfectly sensitive and picks up all the true positives, there would only be a 50% chance of a positive result being true.
Interestingly, the Liverpool data also suggests that the majority of true positive results were in individuals who had higher viral loads. If – and it is an if – higher viral loads are strongly associated with greater infectivity, then these will be the most important asymptomatic cases to identify.
The upshot of this is that antigen testing has some features in its favour for identifying positive cases. The problem is that these benefits may be small if the virus is not prevalent, and they may be massively outweighed by the costs of false reassurance if it is widespread.
Can we avoid false reassurance?
There might be some measures that could potentially reduce these costs. The current messaging that increased testing can “provide reassurance” amplifies the risk of false reassurance, but it could be changed. The advertised purpose of antigen testing could instead be to identify more of the asymptomatic carriers currently flying under the radar.
Some context is important here. In the UK, more accurate testing is currently freely available only for symptomatic individuals and a small number of other groups. This strategy means that many asymptomatic carriers are being missed, and that’s a problem – approximately 40-45% of infections are estimated to be asymptomatic.
It might also be possible to clarify to people that positive results are robust in a way that the negative results are not. We could also impose further restrictions on people with positive results without similarly using negative results as justification for releasing individuals from other existing restrictions.
One problem with all of these strategies is that they are difficult public health messages to communicate. However, the extent of the problem of false reassurance is also determined by the proportion of infectious people among the false negative cases. The Liverpool data suggests an avenue of further study here.
If we could establish firstly that people with low viral loads pose an acceptably low risk of transmission, and secondly that the false negatives generated by antigen tests were restricted to individuals with such low viral loads, then the harm of these false negatives would also be low. We currently lack crucial data to definitively establish these things. However, if we could, then it would support the argument that these tests could still be used as an effective containment strategy, based around highly frequent testing.
There are significant challenges for mitigating the harms of inaccurate mass antigen testing, and a number of other questions remain. But it’s still possible that some form of mass antigen testing could yet be useful in the future.