tag:theconversation.com,2011:/fr/topics/deworming-18815/articlesDeworming – The Conversation2018-10-09T13:59:44Ztag:theconversation.com,2011:article/943632018-10-09T13:59:44Z2018-10-09T13:59:44ZWhy drugs alone won’t stop the spread of parasitic worms in Africa<figure><img src="https://images.theconversation.com/files/238964/original/file-20181002-85602-gdy6el.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=496&fit=clip" /><figcaption><span class="caption">Parasitic worms live in the soil and water where they can constantly reinfect people.
</span> <span class="attribution"><span class="source">EPA/Jane Hahn</span></span></figcaption></figure><p>Deworming has become a common practise across the world. It has proved very successful <a href="http://journals.plos.org/plosmedicine/article?id=10.1371/journal.pmed.0020336">reducing the burden</a> of parasitic worms among children by <a href="https://doi.org/10.1093/infdis/jiu506">more than 50%</a>.</p>
<p>The success of the last 17 years has been driven by deworming initiatives where school-going children get deworming tablets once to twice a year – and could have been greater if there was a vaccine. </p>
<p>But parasitic worms – found mainly in tropical and subtropical areas particularly in sub-Saharan Africa, the Americas, China and East Asia – remain a challenge. The World Health Organisation’s latest figures show that across the world, there are close to <a href="https://end.org/ntds-in-focus/">1.5 billion people</a> – about one fifth of the world’s population – infected with parasitic worms.</p>
<p>The persistence of parasitic worms shows that deworming is not enough to interrupt the worms’ transmission. One part of the problem is that they live in the soil and water where they can constantly reinfect people even after they have been treated. Another part of the worm problem is that diagnoses are poorly efficient, anti-worm drugs are limited and tightly overlapping geographics and ecologics render co-infections with other diseases like HIV, malaria or tuberculosis frequent. </p>
<p>But most importantly, communities are not educated about the dangers. This means that even when they are treated they continue activities that result in them being reinfected. </p>
<p>The WHO has <a href="http://www.who.int/mediacentre/factsheets/fs366/en/">set a goal</a> to eradicate the burden of parasitic worms in children by 2020. To do this, it’s advocating mass <a href="http://journals.plos.org/plosmedicine/article?id=10.1371/journal.pmed.0020336">deworming campaigns</a>. But the only way to achieve the 2020 goal is to combine deworming with other tools such as education campaigns. </p>
<p>We learnt some refreshing lessons in rural Cameroon on how this should be done. <a href="https://www.frontiersin.org/articles/10.3389/fimmu.2018.02295/abstract">Our study</a> looked at a campaign against schistosomiasis – more commonly known as snail fever. Part of the campaign involved educating children, teachers and legal guardians in rural Cameroon about the dangers.</p>
<p>The data we collected showed that mass drug administration alone had a questionable efficiency in halting schistosomiasis transmission. But that coupling mass drug administration of praziquantel to children with information about risk factors of infection might produce better outcomes. In fact, <a href="https://parasitesandvectors.biomedcentral.com/articles/10.1186/s13071-018-2642-x">another recent study</a> in Côte d’Ivoire unequivocally found that repeatedly informing communities how parasitic worms are transmitted and giving them a set of safe practices considerably helped to avoid reinfection. </p>
<p>This underscores the importance of including - among other complementary measures - extensive population awareness campaigns in the deworming strategies to reduce the burden of these diseases. Such health education ventures that are low-cost clearly constitute a necessary tool for creating the enabling environment for mass drug administration campaigns to thrive.</p>
<h2>The challenges</h2>
<p>There are four main challenges when it comes to managing parasitic worms. All are particularly visible in tropical and sub-tropical countries where <a href="http://journals.plos.org/plosntds/article?id=10.1371/journal.pntd.0002651">sanitatory conditions</a> are insufficient and people’s lifestyles undermine attempts to interrupt the transmission of parasitic worms. </p>
<p>The first challenge is diagnosis. The diseases caused by parasitic worms are generally asymptomatic. This means that the early signs of infection are usually missed. In cases where people develop severe disease, they usually <a href="https://www.jci.org/articles/view/34261">have symptoms</a> like diarrhoea, abdominal pain, blood loss that can result in anaemia, cognitive deficit and tissue destruction. On rare occasions, these diseases can result in death.</p>
<p>But because people don’t show symptoms, most do not know that they have the worms in their body until the parasites proliferate. </p>
<p>The second problem is reinfection. Even after populations receive treatment, they can <a href="https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5280090/">still be reinfected</a>. </p>
<p>The third challenge is that there is a limited repertoire of anthelminthic drug therapies. These have been repeatedly administered for decades raising concerns about drug resistance in areas targeted for <a href="https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5280090/">frequent campaigns</a>.</p>
<p>And the fourth is the problem of co-infections with major communicable diseases such as HIV, malaria or tuberculosis. In many instances the parasitic infections accelerate the course of these diseases and worsen their burden. </p>
<h2>Tackling the problem</h2>
<p>Creating awareness in communities is critical and arguably the most cost-effective tool in an anti-worm strategy. </p>
<p>It is important to educate and involve infected people in <a href="https://parasitesandvectors.biomedcentral.com/articles/10.1186/s13071-018-2642-x">mass deworming campaigns</a>. There are some examples of success. Kenya, for example, has managed to eradicate Guinea worm disease as a result of deworming campaigns being <a href="https://www.cartercenter.org/health/guinea_worm/index.html">complemented by health education</a>. Guinea worm disease is caused by <em>Dracunculus medinensis</em> – a long, thread-like worm. </p>
<p>Educational campaigns should include three vital messages. The first must explain how parasitic worms are spread, the second must explain the importance of sanitation in preventing transmission and the third should stress the limitations of mass drug treatments. </p>
<p>These campaigns should include messages on how to protect water sources and sanitation and hygiene <a href="http://www.who.int/water_sanitation_health/publications/wash_standards_school.pdf">practices</a>.</p>
<p>Unless these health awareness campaigns are incorporated into strategies that target parasitic worms, there won’t be much improvement in trying to sustainably reduce the burden of parasitic worms and achieve the elimination rather than the control of these diseases.</p><img src="https://counter.theconversation.com/content/94363/count.gif" alt="The Conversation" width="1" height="1" />
<p class="fine-print"><em><span>Justin Komguep Nono receives funding from the National Research Foundation of South Africa, the Merck Global Health Institute, the International Centre for Genetic Engineering and Biotechnology (CRP/CMR15-05) and the European Union via the European & Developing Countries Clinical Trials Partnership (EDCTP2) programme through funds TMA2016CDF-1571</span></em></p><p class="fine-print"><em><span>Hlumani Ndlovu receives funding from Medical Research Council of South Africa (MRC-SA). </span></em></p>Deworming needs to be accompanied by health awareness campaigns in order to reduce diseases from parasitic worms.Justin Komguep Nono, Research Officer, Institute of Medical Research and Medicinal Plant Studies (Cameroon) and Research fellow, University of Cape TownHlumani Ndlovu, Lecturer, University of Cape TownLicensed as Creative Commons – attribution, no derivatives.tag:theconversation.com,2011:article/451492015-07-28T10:22:13Z2015-07-28T10:22:13ZHalf of biomedical research studies don’t stand up to scrutiny – and what we need to do about that<figure><img src="https://images.theconversation.com/files/89833/original/image-20150727-7646-x6278p.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=496&fit=clip" /><figcaption><span class="caption">How much of the research in these journals could be reproduced?</span> <span class="attribution"><a class="source" href="https://www.flickr.com/photos/yeaki/6961051384">Tobias von der Haar</a>, <a class="license" href="http://creativecommons.org/licenses/by/4.0/">CC BY</a></span></figcaption></figure><p>What if I told you that half of the studies published in scientific journals today – the ones upon which news coverage of medical advances is often based – won’t hold up under scrutiny? You might say I had gone mad. No one would ever tolerate that kind of waste in a field as important – and expensive, to the tune of roughly <a href="http://officeofbudget.od.nih.gov/pdfs/FY15/FY2015_Overview.pdf">US$30 billion in federal spending per year</a> – as biomedical research, right? After all, this is the crucial work that hunts for explanations for diseases so they can better be treated or even cured.</p>
<p>Wrong. The rate of what is referred to as “irreproducible research” – more on what that means in a moment – exceeds 50%, <a href="http://dx.doi.org/10.1371/journal.pbio.1002165">according to a recent paper</a>. Some <a href="http://dx.doi.org/10.1371/journal.pmed.0020124">estimates are even higher</a>. In one analysis, just <a href="http://dx.doi.org/10.1038/483531a">11% of preclinical cancer research studies could be confirmed</a>. That means that an awful lot of “promising” results aren’t very promising at all, and that a lot of researchers who could be solving critical problems based on previously published work end up just spinning their wheels.</p>
<p>So what gives? And how can we fix this problem?</p>
<figure class="align-center zoomable">
<a href="https://images.theconversation.com/files/89835/original/image-20150727-7662-j5cbjp.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=1000&fit=clip"><img alt="" src="https://images.theconversation.com/files/89835/original/image-20150727-7662-j5cbjp.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=754&fit=clip" srcset="https://images.theconversation.com/files/89835/original/image-20150727-7662-j5cbjp.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=600&h=377&fit=crop&dpr=1 600w, https://images.theconversation.com/files/89835/original/image-20150727-7662-j5cbjp.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=600&h=377&fit=crop&dpr=2 1200w, https://images.theconversation.com/files/89835/original/image-20150727-7662-j5cbjp.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=600&h=377&fit=crop&dpr=3 1800w, https://images.theconversation.com/files/89835/original/image-20150727-7662-j5cbjp.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=754&h=473&fit=crop&dpr=1 754w, https://images.theconversation.com/files/89835/original/image-20150727-7662-j5cbjp.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=754&h=473&fit=crop&dpr=2 1508w, https://images.theconversation.com/files/89835/original/image-20150727-7662-j5cbjp.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=754&h=473&fit=crop&dpr=3 2262w" sizes="(min-width: 1466px) 754px, (max-width: 599px) 100vw, (min-width: 600px) 600px, 237px"></a>
<figcaption>
<span class="caption">Hmmm, I didn’t expect those results….</span>
<span class="attribution"><a class="source" href="http://www.shutterstock.com/pic-64872892/stock-photo-chemistry-recipient-with-ink-color-inside.html">Test tubes image via www.shutterstock.com</a></span>
</figcaption>
</figure>
<h2>What worms tell us about reproducibility</h2>
<p>Although definitions of reproducibility and replication vary somewhat, for a study to be reproducible, another researcher needs to be able to replicate it, meaning use the same data and analysis to come to the same conclusions. There are lots of reasons why a study may not pass the replication test, from flat-out errors to a failure to adequately describe the methodology used. A researcher may have forgotten about a step in the process when he wrote up the methodology, for example, counted data in the wrong category, or written the wrong code for her statistics program.</p>
<p><a href="https://theconversation.com/clearing-the-air-why-more-retractions-are-good-for-science-6008">Faking results</a> is another reason, but it’s not nearly as common as others. Out-and-out fraud like that, or suspected fraud, is the reason for a bit <a href="http://dx.doi.org/10.1073/pnas.1212247109">fewer than half of the 400-plus retractions per year</a>. But there are something like two million papers published annually, so the vast majority of studies containing irreproducible data are never retracted. And most scientists would agree that they shouldn’t be; after all, most science is overturned one way or another over time. Retraction should be reserved for the most severe cases. That doesn’t mean irreproducible papers shouldn’t be somehow marked, though.</p>
<figure class="align-right zoomable">
<a href="https://images.theconversation.com/files/89838/original/image-20150727-7646-utfyxu.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=1000&fit=clip"><img alt="" src="https://images.theconversation.com/files/89838/original/image-20150727-7646-utfyxu.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=237&fit=clip" srcset="https://images.theconversation.com/files/89838/original/image-20150727-7646-utfyxu.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=600&h=450&fit=crop&dpr=1 600w, https://images.theconversation.com/files/89838/original/image-20150727-7646-utfyxu.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=600&h=450&fit=crop&dpr=2 1200w, https://images.theconversation.com/files/89838/original/image-20150727-7646-utfyxu.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=600&h=450&fit=crop&dpr=3 1800w, https://images.theconversation.com/files/89838/original/image-20150727-7646-utfyxu.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=754&h=566&fit=crop&dpr=1 754w, https://images.theconversation.com/files/89838/original/image-20150727-7646-utfyxu.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=754&h=566&fit=crop&dpr=2 1508w, https://images.theconversation.com/files/89838/original/image-20150727-7646-utfyxu.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=754&h=566&fit=crop&dpr=3 2262w" sizes="(min-width: 1466px) 754px, (max-width: 599px) 100vw, (min-width: 600px) 600px, 237px"></a>
<figcaption>
<span class="caption">A girl takes her deworming tablet.</span>
<span class="attribution"><a class="source" href="https://www.flickr.com/photos/savethechildrenusa/7051746493">Save the Children</a>, <a class="license" href="http://creativecommons.org/licenses/by-nc-nd/4.0/">CC BY-NC-ND</a></span>
</figcaption>
</figure>
<p>Here’s a fresh example of a study that turned out not to be reproducible, because the results couldn’t be replicated: as <a href="http://www.buzzfeed.com/bengoldacre/deworming-trials">Ben Goldacre relates in BuzzFeed</a>, two economists published a <a href="http://dx.doi.org/10.1111/j.1468-0262.2004.00481.x">massive study in 2004</a> claiming that a “deworm everyone” approach in Kenya “improved children’s health, school performance, and school attendance,” even among children several miles away who didn’t get deworming pills. <a href="http://www.who.int/elena/titles/deworming/en/">Endorsed by the World Health Organization</a>, it helped set policy that affects hundreds of millions of children annually in the developing world.</p>
<p>But now researchers have published <a href="http://dx.doi.org/10.1093/ije/dyv127">papers</a> describing two <a href="http://dx.doi.org/10.1093/ije/dyv128">failures</a> to replicate the original findings. Many of them just didn’t hold up, although some did.</p>
<p>That, as Goldacre explains, “is definitely problematic.” But the reanalyses were possible only because the original authors “had the decency, generosity, strength of character, and intellectual confidence to let someone else peer under the bonnet” – a <a href="http://dx.doi.org/10.1001/jama.2014.9646">rare situation indeed</a>.</p>
<h2>The fixes</h2>
<p>Researchers are aware of the reproducibility problem, and some are trying to fix it. In response to alarming findings about the reproducibility of <a href="http://dx.doi.org/10.1038/483531a">basic cancer research</a>, a program called the <a href="http://validation.scienceexchange.com/#/reproducibility-initiative">Reproducibility Initiative</a> has started providing “both a mechanism for scientists to independently replicate their findings and a reward for doing so.” It’s <a href="http://blog.scienceexchange.com/2012/08/the-reproducibility-initiative/">chosen 50 studies for independent validation</a> – or not, since there’s certainly a chance the initial results won’t be reproducible. Those working on the project will perform the same kind of analyses that researchers did in the worm study replications. A similar effort has been <a href="https://osf.io/ezcuj/wiki/home/">ongoing in psychology</a>, and other projects are under way in the <a href="http://www.bitss.org/">social sciences</a>.</p>
<figure class="align-center zoomable">
<a href="https://images.theconversation.com/files/89836/original/image-20150727-7668-esb8u0.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=1000&fit=clip"><img alt="" src="https://images.theconversation.com/files/89836/original/image-20150727-7668-esb8u0.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=754&fit=clip" srcset="https://images.theconversation.com/files/89836/original/image-20150727-7668-esb8u0.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=600&h=335&fit=crop&dpr=1 600w, https://images.theconversation.com/files/89836/original/image-20150727-7668-esb8u0.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=600&h=335&fit=crop&dpr=2 1200w, https://images.theconversation.com/files/89836/original/image-20150727-7668-esb8u0.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=600&h=335&fit=crop&dpr=3 1800w, https://images.theconversation.com/files/89836/original/image-20150727-7668-esb8u0.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=754&h=421&fit=crop&dpr=1 754w, https://images.theconversation.com/files/89836/original/image-20150727-7668-esb8u0.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=754&h=421&fit=crop&dpr=2 1508w, https://images.theconversation.com/files/89836/original/image-20150727-7668-esb8u0.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=754&h=421&fit=crop&dpr=3 2262w" sizes="(min-width: 1466px) 754px, (max-width: 599px) 100vw, (min-width: 600px) 600px, 237px"></a>
<figcaption>
<span class="caption">Research data need to be an open book.</span>
<span class="attribution"><a class="source" href="https://www.flickr.com/photos/brenda-starr/5813347420">Brenda Clarke</a>, <a class="license" href="http://creativecommons.org/licenses/by/4.0/">CC BY</a></span>
</figcaption>
</figure>
<p>All of these efforts will require scientists to share data, as the authors of the deworming study did. That has been a requirement in human studies for some years now, by <a href="http://www.nhlbi.nih.gov/research/funding/human-subjects/data-sharing">many funders</a>, and it’s <a href="http://www.icmje.org/recommendations/browse/publishing-and-editorial-issues/clinical-trial-registration.html">encouraged by many journal editors</a>. And while it’s not met 100% of the time, <a href="http://dx.doi.org/10.1056/NEJMsa1409364">compliance is growing</a>. Some basic science journals are <a href="http://www.nytimes.com/2015/06/26/science/journal-science-releases-guidelines-for-publishing-scientific-studies.html?_r=0">moving to make it a requirement</a>, too.</p>
<p>Perhaps more important, however, is that researchers – and the public that funds many of them – realize that science is a process, and that all knowledge is provisional. “It’s not just naive to expect that all research will be perfectly free from errors,” writes Goldacre, “it’s actively harmful.” <a href="http://www.vox.com/2015/3/23/8264355/research-study-hype">Journalists, take note</a>.</p>
<p>Translated into policy, that means valuing replication efforts, which right now are essentially unfunded and hardly ever published. If we want scientists to validate others’ work, we’ll need to create grants to do that. That means digging up additional funding, but replicating a study costs a tiny fraction of what the original work does. Funding new studies based on those that turn out to be irreproducible…well, now that’s expensive.</p><img src="https://counter.theconversation.com/content/45149/count.gif" alt="The Conversation" width="1" height="1" />
<p class="fine-print"><em><span>Ivan Oransky, global editorial director of MedPage Today, is co-founder of Retraction Watch. Retraction Watch, through its parent organization, The Center For Scientific Integrity, is funded by a generous grant from the John D. and Catherine T. MacArthur Foundation.</span></em></p>It’s a problem when much of what winds up in scientific journals isn’t replicable, for various reasons. The research community is taking baby steps toward addressing the “reproducibility crisis.”Ivan Oransky, Distinguished Writer In Residence, Arthur Carter Journalism Institute, New York UniversityLicensed as Creative Commons – attribution, no derivatives.tag:theconversation.com,2011:article/450962015-07-23T17:34:46Z2015-07-23T17:34:46ZThe positive impact of deworming in Kenyan schools: the evidence untangled<figure><img src="https://images.theconversation.com/files/89519/original/image-20150723-22806-zzjob.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=496&fit=clip" /><figcaption><span class="caption">A health worker dispenses albendazole tablets to a child on National Deworming Day in Kisumu, Kenya. </span> <span class="attribution"><span class="source">Evidence Action, Courtesy of Photoshare</span></span></figcaption></figure><p>A <a href="http://ije.oxfordjournals.org/content/early/2015/07/21/ije.dyv128.abstract">re-analysis</a> of research on school-based deworming in Kenya <a href="http://ije.oxfordjournals.org/content/early/2015/07/21/ije.dyv129.short?rss=1">strongly supports</a> the finding that treatment improves school attendance of both children who are treated and those who are not. </p>
<p>The re-analysis confirms the core findings of our original <a href="http://onlinelibrary.wiley.com/doi/10.1111/j.1468-0262.2004.00481.x/abstract">research</a>. These were that: </p>
<ul>
<li><p>Deworming programmes reduced school absenteeism in treatment schools by one-quarter,</p></li>
<li><p>It was cheaper than other ways of increasing school participation, and </p></li>
<li><p>Appeared to improve school attendance in schools where no children were treated. </p></li>
</ul>
<p>The re-analysis also corrects a few errors, some mislabelling and clarifies a few data issues in the original research. As authors of the original study, our <a href="http://ije.oxfordjournals.org/content/early/2015/07/21/ije.dyv129.short?rss=1">assessment</a> of the re-analysis is that it provides a welcome endorsement of the efficacy of school-based deworming interventions. We do, however, disagree with some of its findings.</p>
<p>Deworming is currently being implemented as policy in many parts of the <a href="http://wber.oxfordjournals.org/content/early/2015/06/03/wber.lhv008.abstract">developing world</a>. Recent estimates suggest that 280 million children, out of 870 million in need, are treated for worms, many via school-based and community-based programmes. This focus on deworming was triggered, in large part, by the findings of randomised controlled trials conducted by a team of economists, including myself, between 1998 and 2004.</p>
<p>That the core of these findings has been confirmed by epidemiologists underscores the importance of the approach adopted by policymakers like the World Health Organisation (WHO). It has recommended mass treatment once a year in regions where worm prevalence is 20% and twice a year in regions where worm it is 50%. Our recently published <a href="http://wber.oxfordjournals.org/content/early/2015/06/03/wber.lhv008.abstract">paper</a> suggests that the WHO recommendations are justified on human rights, welfare economics and cost-effectiveness grounds.</p>
<h2>Areas of agreement and disagreement</h2>
<p>The re-analysis consists of two papers. The first is a pure replication exercise. This is useful. The differences are relatively minor, and the bottom-line policy conclusions remain largely intact.</p>
<p>Policies have been developed based on three main findings in <a href="http://onlinelibrary.wiley.com/doi/10.1111/j.1468-0262.2004.00481.x/abstract">our earlier research</a>. </p>
<ul>
<li><p>First, that school-based deworming in Kenya led to large drops in worm infections for those taking the tablets and for other community members by breaking the cycle of transmission.</p></li>
<li><p>Second, that school attendance rose sharply in the deworming treatment schools over the two years of the study. </p></li>
<li><p>Third, since deworming is cheap, it achieves these two previous goals very cost effectively. The study has figured into recent policy debates about the attractiveness of mass deworming in several countries, including Kenya.</p></li>
</ul>
<p>We differ with one aspect of the pure replication report. We disagree with the interpretation of the people who did not necessarily take the drugs themselves, but benefited even though they lived within 3kms to 6kms of treatment schools.</p>
<p>The updated estimates are too <a href="http://dx.doi.org/10.1093/ije/dyv128">“noisy”</a> to be useful. What we mean by this is that when the data is updated, the statistical estimates are not informative, and neither positive nor negative values can be ruled out within a wide range. Our interpretation is that it is not worth putting much weight on these estimates but the replication authors use these uninformative figures to cast doubt on all other results in the paper. This is inappropriate. </p>
<p>The second part of the re-analysis is a statistical replication which assesses the “robustness” of the 2004 findings. This is a common goal of researchers – figuring out how much conclusions change when assumptions are changed.</p>
<p>We identified more substantive problems with the statistical replication which contains a number of <a href="http://dx.doi.org/10.1093/ije/dyv128">analytical errors</a> and draws conclusions that are contradicted by the data.</p>
<p>Of particular concern are the questions raised about the robustness of the original research result that deworming has benefits for school attendance. To assess robustness in our original research, we considered statistical models, samples, approaches to weighting data, and definitions of deworming “treatment”. </p>
<p>Questions about the robustness of the result is based on two puzzling analytical choices neither of which can be justified with data. These are:</p>
<ul>
<li><p>Incorrectly defining the deworming treatment variable of when the children received their treatment. The re-analysis suggest ambiguity around treatment dates. But there was none. The original research paper, data and project documentation are all crystal clear. </p></li>
<li><p>The most glaring error is that they chose to split the data into separate years. In doing so they uncovered “unexpected” patterns in the data, regarding the correlation between deworming treatment in a school and the number of attendance observations collected in that school. They argue this pattern could “bias” the analysis. </p></li>
</ul>
<p>We directly tested whether these “unexpected” patterns existed in the data and conclude that they don’t. We continue to believe that in the absence of any justification for splitting the data set, the analysis of the Kenya deworming data should be carried out on the full one. </p>
<p>This leads to the robust conclusion that deworming improved school attendance in the rural schools that we studied. These results contribute to a growing research literature finding large positive long-run impacts of deworming on educational and labour market outcomes.</p><img src="https://counter.theconversation.com/content/45096/count.gif" alt="The Conversation" width="1" height="1" />
<p class="fine-print"><em><span>Edward Miguel does not work for, consult, own shares in or receive funding from any company or organisation that would benefit from this article, and has disclosed no relevant affiliations beyond their academic appointment.</span></em></p>A re-analysis of research into deworming interventions at Kenyan schools has confirmed some findings and disputed others. However, it does not take away from the programme’s effectiveness.Edward Miguel, Oxfam Professor of Environmental and Resource Economics, University of California, BerkeleyLicensed as Creative Commons – attribution, no derivatives.tag:theconversation.com,2011:article/449262015-07-23T04:21:32Z2015-07-23T04:21:32ZHow re-analysing the data of scientific research can change the findings<figure><img src="https://images.theconversation.com/files/89217/original/image-20150721-24258-1ikoni1.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=496&fit=clip" /><figcaption><span class="caption">A re-analysis of data of deworming at schools in Kenya has generated different findings.</span> <span class="attribution"><span class="source">2014 Evidence Action/Photoshare</span></span></figcaption></figure><p>Science, many people believe, is kept in check by scientists reviewing each other’s work. This has recently extended to re-analysis of data to see if results can be replicated, and has overturned important findings in <a href="http://news.sciencemag.org/health/2014/04/armed-new-data-researchers-again-challenge-effectiveness-antiflu-drug">medicine</a>, <a href="http://www.economist.com/news/finance-and-economics/21576362-seminal-analysis-relationship-between-debt-and-growth-comes-under">economics</a>, and <a href="http://today.uconn.edu/2015/05/study-negative-findings-for-children-of-gay-parents-dont-hold-up-to-scrutiny/">sociology</a>. </p>
<p>We re-analysed an influential randomised controlled <a href="http://www.ebbp.org/course_outlines/randomized_controlled_trials/">trial</a> of deworming in Kenyan schools. We found that even for a randomised controlled trial – lauded as the most robust method to identify impact – there are aspects of analysis and reporting where re-analysis can shed new light.</p>
<p>Re-analysis is a powerful tool in the review of important studies, and should be supported with data made available by researchers and with adequate funding. The publication of the <a href="http://dx.doi.org/10.1093/ije/dyv128">replication</a> <a href="http://dx.doi.org/10.1093/ije/dyv127">results</a> by the International Journal of Epidemiology is a watershed. To our knowledge it is very unusual for an international journal to publish a re-analysis of an already published trial.</p>
<p>The original authors have <a href="http://ije.oxfordjournals.org/content/early/2015/07/21/ije.dyv129.short?rss=1">responded</a> to our re-analysis, which is also <a href="https://theconversation.com/the-positive-impact-of-deworming-in-kenyan-schools-the-evidence-untangled-45096">published</a> in the journal today. </p>
<h2>Randomised controlled trials</h2>
<p>Randomised controlled trials were primarily developed in medicine, agriculture and psychology. They evaluate programme effects by comparing a group of individuals randomly allocated to get the service with a group who were not, using data collected some time after the intervention started. </p>
<p>Recently, the number of randomised controlled trials has <a href="http://www.nber.org/papers/w15794">exploded</a> in development economics. This has caught people’s attention. Economists doing trials have appeared in the Time Magazine’s <a href="http://content.time.com/time/specials/packages/article/0,28804,2066367_2066369_2066106,00.html">100 most influential people list</a>, won major <a href="http://www.ssrc.org/hirschman/2014-statement-of-commendation">awards</a> and shifted the <a href="http://www.economist.com/news/finance-and-economics/21591573-once-treated-scorn-randomised-control-trials-are-coming-age-random-harvest">development discourse</a>.</p>
<p>Much credit for sparking this movement goes to Michael Kremer and Ted Miguel’s <a href="http://onlinelibrary.wiley.com/doi/10.1111/j.1468-0262.2004.00481.x/abstract">paper</a> on deworming effects on school attendance in Kenya. </p>
<p>A quarter of the world’s population is infected with <a href="http://www.povertyactionlab.org/evaluation/primary-school-deworming-kenya">intestinal worms</a>, including hookworm, roundworm, schistosomiasis and whipworm. It is particularly prevalent among school-aged children in developing countries. The effects of intestinal worms are especially pronounced in Africa, where nearly half of the total disease burden is due to infectious and parasitic diseases, including worm infections. </p>
<p>Using a trial design now known as a [“stepped-wedge”](http://www.contemporaryclinicaltrials.com/article/S1551-7144(06\00063-2/abstract), Kremer and Miguel reported the benefits of a programme of health education and mass administration of <a href="http://www.givewell.org/international/technical/programs/deworming">deworming drugs</a>. </p>
<p>There were several results that proved especially influential. First, they found that the deworming programme reduced school absenteeism in treatment schools by one-quarter. It was cheaper than other ways of increasing school participation. </p>
<p>It also appeared to improve school attendance in schools where no children were treated. This is plausible since if there are fewer worms around, other children may be less likely to get infected. </p>
<p>The results led many, including the <a href="http://www.copenhagenconsensus.com/copenhagen-consensus-iii/outcome">Copenhagen Consensus in 2012</a>, to conclude that deworming is one of the most cost-effective development interventions. The organisation’s Deworm the World initiative received strong independent <a href="http://www.givewell.org/international/top-charities/deworm-world-initiative">endorsement</a> and <a href="http://www.evidenceaction.org/blog-full/largest-deworming-program-in-india-to-start-with-support-from-evidence-action">support</a> from governments.</p>
<h2>A fresh look at the data</h2>
<figure class="align-right ">
<img alt="" src="https://images.theconversation.com/files/89225/original/image-20150721-24301-bs1svv.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=237&fit=clip" srcset="https://images.theconversation.com/files/89225/original/image-20150721-24301-bs1svv.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=600&h=314&fit=crop&dpr=1 600w, https://images.theconversation.com/files/89225/original/image-20150721-24301-bs1svv.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=600&h=314&fit=crop&dpr=2 1200w, https://images.theconversation.com/files/89225/original/image-20150721-24301-bs1svv.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=600&h=314&fit=crop&dpr=3 1800w, https://images.theconversation.com/files/89225/original/image-20150721-24301-bs1svv.jpg?ixlib=rb-1.1.0&q=45&auto=format&w=754&h=395&fit=crop&dpr=1 754w, https://images.theconversation.com/files/89225/original/image-20150721-24301-bs1svv.jpg?ixlib=rb-1.1.0&q=30&auto=format&w=754&h=395&fit=crop&dpr=2 1508w, https://images.theconversation.com/files/89225/original/image-20150721-24301-bs1svv.jpg?ixlib=rb-1.1.0&q=15&auto=format&w=754&h=395&fit=crop&dpr=3 2262w" sizes="(min-width: 1466px) 754px, (max-width: 599px) 100vw, (min-width: 600px) 600px, 237px">
<figcaption>
<span class="caption">A health worker dispenses albendazole tablets to a child on National Deworming Day in Kisumu, Kenya.</span>
<span class="attribution"><span class="source">Evidence Action, Courtesy of Photoshare</span></span>
</figcaption>
</figure>
<p>The decision to re-analyse this research was based on broader methodological questions. There was not a particular interest in deworming. It was a highly influential study published by economists in a format unfamiliar to epidemiologists. </p>
<p>We suspected that the disconnect between how highly it has been praised by other economists and how it had been previously reviewed by epidemiologists working on <a href="http://www.cochrane.org/">Cochrane reviews</a> reflected differences in analysis and reporting between the two fields. The re-analysis was intended to help bridge the differences between epidemiology and economics. </p>
<p>We re-ran the original analyses using files sent to us by Miguel and his team. While few mistakes were found, the authors erroneously reported evidence of effect of the intervention on anaemia. But there was no evidence of this in reanalysis.</p>
<p>There were also coding errors which meant that the total effect on school attendance in the paper was overestimated. There was little evidence that attendance was improved in untreated schools.</p>
<p>Then we re-analysed the data using the methods that we, as epidemiologists, would typically use for analysing trials of this kind. Our analysis showed some evidence that the intervention was associated with greater school attendance. However, we add the important caveat that this conclusion is at high risk of bias.</p>
<p>Bias often comes into a study because of ways participants are selected for measurement, including if they are lost after first being contacted, and how good those measurements are. Bias can creep in at all stages of the research process, and usually despite the best efforts of the researchers. </p>
<p>Our concern about the bias was triggered by uncertainty about how the school attendance data was collected. There is a possibility that the way in which the data was collected might have been systematically different in schools that did and did not get the deworming treatment. </p>
<p>Although it is not known if this is the case, the amount of missing data and some peculiar patterns found in the re-analysis gave us some cause for concern. And since neither anaemia, weight-for-age, nor height-for-age appear to have been affected by the treatment, we have no other evidence for any intermediate steps on a causal chain between deworming and school attendance that might have offset some of our concerns about bias.</p>
<h2>The future of re-analysis</h2>
<p>This deworming re-analysis exercise should not directly affect deworming programmes. Another group is updating a systematic review and will consider the wider evidence. But it shows that systematic reviews are the best way to inform policy, and <a href="http://www.vox.com/2015/3/23/8264355/research-study-hype">avoid hype</a>.</p>
<p>The explosion of randomised trials in development is exciting. Trials in this area could be improved by adopting standards from the medical field. This is already happening: pre-analysis plans are becoming more common, and have been <a href="http://cega.berkeley.edu/events/pre-analysis-plans/">championed</a> by economists at Berkeley – a movement led by, among others, Ted Miguel. </p>
<p>Adopting – or adapting – the medical field’s <a href="http://www.consort-statement.org/">standards</a> for trial reporting may lead to more transparent conclusions and reveal risks of bias. These standards address the fact that when it comes to randomised trials, especially in low income settings, the quality of data and measurements are just as important as the statistics subsequently applied to them.</p>
<p>We anticipate that re-analysis will become more common, will improve transparency, accountability, and strengthen the literature that policymakers use to base decisions that affect the health and happiness of millions of people around the world.</p><img src="https://counter.theconversation.com/content/44926/count.gif" alt="The Conversation" width="1" height="1" />
<p class="fine-print"><em><span>The authors do not work for, consult, own shares in or receive funding from any company or organisation that would benefit from this article, and have disclosed no relevant affiliations beyond their academic appointment.</span></em></p>The re-analysis of data can provide valuable new findings and it can improve transparency, accountability, and strengthen the literature that policymakers base their decisions on.James Hargreaves, Director, Centre for Evaluation, London School of Hygiene & Tropical MedicineCalum Davey, Research Fellow in Epidemiology , London School of Hygiene & Tropical MedicineLicensed as Creative Commons – attribution, no derivatives.