A re-analysis of research on school-based deworming in Kenya strongly supports the finding that treatment improves school attendance of both children who are treated and those who are not.
The re-analysis confirms the core findings of our original research. These were that:
Deworming programmes reduced school absenteeism in treatment schools by one-quarter,
-
It was cheaper than other ways of increasing school participation, and
Appeared to improve school attendance in schools where no children were treated.
The re-analysis also corrects a few errors, some mislabelling and clarifies a few data issues in the original research. As authors of the original study, our assessment of the re-analysis is that it provides a welcome endorsement of the efficacy of school-based deworming interventions. We do, however, disagree with some of its findings.
Deworming is currently being implemented as policy in many parts of the developing world. Recent estimates suggest that 280 million children, out of 870 million in need, are treated for worms, many via school-based and community-based programmes. This focus on deworming was triggered, in large part, by the findings of randomised controlled trials conducted by a team of economists, including myself, between 1998 and 2004.
That the core of these findings has been confirmed by epidemiologists underscores the importance of the approach adopted by policymakers like the World Health Organisation (WHO). It has recommended mass treatment once a year in regions where worm prevalence is 20% and twice a year in regions where worm it is 50%. Our recently published paper suggests that the WHO recommendations are justified on human rights, welfare economics and cost-effectiveness grounds.
Areas of agreement and disagreement
The re-analysis consists of two papers. The first is a pure replication exercise. This is useful. The differences are relatively minor, and the bottom-line policy conclusions remain largely intact.
Policies have been developed based on three main findings in our earlier research.
First, that school-based deworming in Kenya led to large drops in worm infections for those taking the tablets and for other community members by breaking the cycle of transmission.
Second, that school attendance rose sharply in the deworming treatment schools over the two years of the study.
Third, since deworming is cheap, it achieves these two previous goals very cost effectively. The study has figured into recent policy debates about the attractiveness of mass deworming in several countries, including Kenya.
We differ with one aspect of the pure replication report. We disagree with the interpretation of the people who did not necessarily take the drugs themselves, but benefited even though they lived within 3kms to 6kms of treatment schools.
The updated estimates are too “noisy” to be useful. What we mean by this is that when the data is updated, the statistical estimates are not informative, and neither positive nor negative values can be ruled out within a wide range. Our interpretation is that it is not worth putting much weight on these estimates but the replication authors use these uninformative figures to cast doubt on all other results in the paper. This is inappropriate.
The second part of the re-analysis is a statistical replication which assesses the “robustness” of the 2004 findings. This is a common goal of researchers – figuring out how much conclusions change when assumptions are changed.
We identified more substantive problems with the statistical replication which contains a number of analytical errors and draws conclusions that are contradicted by the data.
Of particular concern are the questions raised about the robustness of the original research result that deworming has benefits for school attendance. To assess robustness in our original research, we considered statistical models, samples, approaches to weighting data, and definitions of deworming “treatment”.
Questions about the robustness of the result is based on two puzzling analytical choices neither of which can be justified with data. These are:
Incorrectly defining the deworming treatment variable of when the children received their treatment. The re-analysis suggest ambiguity around treatment dates. But there was none. The original research paper, data and project documentation are all crystal clear.
The most glaring error is that they chose to split the data into separate years. In doing so they uncovered “unexpected” patterns in the data, regarding the correlation between deworming treatment in a school and the number of attendance observations collected in that school. They argue this pattern could “bias” the analysis.
We directly tested whether these “unexpected” patterns existed in the data and conclude that they don’t. We continue to believe that in the absence of any justification for splitting the data set, the analysis of the Kenya deworming data should be carried out on the full one.
This leads to the robust conclusion that deworming improved school attendance in the rural schools that we studied. These results contribute to a growing research literature finding large positive long-run impacts of deworming on educational and labour market outcomes.