UK United Kingdom

Flawed sting operation singles out open access journals

In a sting operation, John Bohannon, a correspondent of Science, claims to have exposed dodgy open access journals. His argument seems to be that, because of their business model, some journals are biased…

Isn’t open access better? fuzzcaminski

In a sting operation, John Bohannon, a correspondent of Science, claims to have exposed dodgy open access journals. His argument seems to be that, because of their business model, some journals are biased towards accepting scientific articles, regardless of their quality. Sadly, Bohannon’s operation adds little to what we already know.

Much new knowledge, often created by the use of taxpayers' money, is locked behind paywalls of subscription-based journals. But there is a growing movement to make this knowledge freely available. Researchers around the world are being urged to publish in open access journals, and many are seeking to do so.

Where there is demand, the market learns to supply. This has come in form of thousands of open access journal publishers. But in this rapid rise, the journal Science finds in a sting operation that many such journals will accept anything in the hope of earning some money.

For the sting operation, Bohannon confected a scientific paper and sent it to more than 300 journals listed in the Directory of Open Access Journals. Some of these journals were also on Beall’s list, managed by Jeffrey Beall, a library scientist at the University of Colorado, Denver, to weed out the bad ones.

More than half of those journals accepted the bogus manuscript, which was riddled with ethical approval problems and scientific anomalies. Many of these publishers misrepresented their locations. They all claimed to be peer-reviewed journals, but many of them didn’t bother to do any sort of review. A few provided reviewer comments, but agreed to publish the article even when the reviewers raised serious questions.

Bohannon claims these journals would publish anything, because in the “author pays gold model” the publisher makes money only when the article is published.

It would seem that Bohannon has neatly demonstrated a fatal flaw in open access publishing. Bohannon never explicitly compares open access model to the subscription model (in which the researcher submitting the article doesn’t pay but those reading it do), but his hypothesis seems to be that open access journals driven by publication charges will be inherently biased towards acceptance.

On the surface this looks like a potentially deadly blow to open access journals that levy Article Processing Charges (APCs). But I found some glaring problems with the article and its premises.

In short, Bohannon’s article isn’t really about open access. It’s about a flawed system of trusting journals and the inherent problems in peer review, but he targets only open access here.

Problems of this nature have been explored before. For instance, in a 1982 paper, researchers submitted articles that a journal had already published under different author names. Nearly all the submissions were rejected. Many reviewers described the manuscript as having “serious methodological flaw”.

Does Science do science?

Bohannon’s methodology in the sting operation is questionable. He goes to great lengths to concoct a bogus methodology for submission and to ensure the scientific flaws in his paper were credible. The key problem, however, lies in two sentences, tucked away in Bohannon’s coda. He writes:

Some say that the open-access model itself is not to blame for the poor quality control revealed by Science’s investigation. “If I had targeted traditional, subscription-based journals,” [David] Roos told me, “I strongly suspect you would get the same result.”

Sadly, these two sentences appear at the very end of the article. All along it seems to be an attack on open access journals. While it is not directly mentioned, the implication is “it would never happen in the subscription model”. But, given this wasn’t tested, how do we know?

Value addition in the publication process happens at the peer review stage, as most journals claim. The journals exposed were clearly not adding that value. But this value addition is the same whether the journal is open access or not.

So, why did Science publish such a clearly incomplete study? The harsh truth is that Bohannon’s article is hostile. He submitted articles only to open access journals. This omission then wrongly links the failure of deeper problems in academia to a single business model.

While he acknowledges that the top players (including the journal PLOS ONE) provided rigorous review, Bohannon submitted his bogus paper mostly to poor journals. They do not represent open access as a whole. Although Bohannon argues that “open access has multiplied that underclass of journals”, I would like to counter that it is only through a history of masking editorial processes amid claims of “value added” that we have arrived at this mess.

Peer review is a function of academic labour. Editorial decisions should be made by qualified and respected academics who run journals. If we used this model, not even the most gullible of authors would attempt submission to this underclass of journals and their predatory behaviour wouldn’t exist.

Update: Following a correction from Science, David Roos’s quote has been edited. Details in Roos’s comment below.

Sign in to Favourite

Join the conversation

17 Comments sorted by

  1. Pandelis Perakakis

    Postdoctoral Researcher

    As Martin correctly notes, Peer review is the most important value added by journals, and although it costs close to zero to publishers, as both editors and reviewers work for free, it is what justifies subscription/publication fees. In this way, peer review lies at the heart of the current scientific publishing model and we can't expect any serious publishing reform without addressing the problems associated with how science and individual scientists are evaluated.

    The <a href="">LIBRE</a>; project was specifically developed by <a href="">Open Scholar</a> to propose an alternative peer review process, independent of academic journals and managed by authors themselves in an open and transparent manner. Importantly, it is a complementary evaluation process that does not require authors to submit to specific journals, and that fosters openess and collaboration for the benefit of science and society.

  2. Michel Hockx

    Professor of Chinese at SOAS, University of London

    In the "author pays gold model," the money used to pay for open access publication still comes from the taxpayer, through increased research funding to universities in order to cover the cost of open access publishing. Otherwise, publishing success would depend on which academic happens to have more private money.

    1. Ken Friedman

      University Distinguished Professor at Swinburne University of Technology

      In reply to Gavin Moodie

      This seems to me to be an inaccurate criticism of the journal Science. Science is not a commercial publisher in the ordinary sense that Elsevier or Sage are commercial enterprises. Science is owned and published by a not-for-profit association, the American Association for the Advancement of Science. It is, in this sense, something like a university press or a journal published by a research center.

      With respect to the criticism itself, it's worth reading the John Bohannon article and the entire…

      Read more
    2. Gavin Moodie
      Gavin Moodie is a Friend of The Conversation.

      Adjunct professor at RMIT University

      In reply to Ken Friedman

      Some university presses also have a direct commercial interest in traditional forms of publishing, and indeed, there is a split between the traditional Australian university presses which attack open access publishing and the newer university presses which publish online and provide open access to at least some of their publications.

      The journal Science clearly has a commercial interest in the traditional subscription model of journal publishing which it is not affected by it allocating surpluses to its objects rather than distributing them to shareholders.

    3. Ken Friedman

      University Distinguished Professor at Swinburne University of Technology

      In reply to Gavin Moodie

      One must distinguish between an "interest" and "commercial interest."

      I support many forms of publishing, including open access. The model of the university press is very much to the point here. Excellent open access publishers such as Open Humanities Press, Open Book Publishing, or the Interaction Design Foundation (on whose board I serve) use a full free open access model, some with print-on-demand capacity. I also edit a series for The MIT Press, and I can say that we are not interested in the kinds of activities that define commercial publishing. The Press must sell books to break even and add new books, but the publisher puts its profits back into the book business, and not into commerce.

      The fact that some university presses oppose open access publishing does not make them commercial. I disagree with their position, but I don't see that they hold a commercial objective in maintaining it.

    4. Gavin Moodie
      Gavin Moodie is a Friend of The Conversation.

      Adjunct professor at RMIT University

      In reply to Ken Friedman

      I suggest that different university presses have different positions. MIT has a laudable commitment to open scholarship. But the chief executive officer of Melbourne University Press Louise Adler and some other traditional Australian university presses have long opposed open access for what I read to be straight commercial reasons.

      In a piece published in the Weekend Australian on 14 September Adler (2013) opens with: 'Books and the business of books are imperiled as never before' and spends…

      Read more
  3. Chris Booker

    Research scientist

    This article has really made quite an impact in the blogosphere, there's some interesting academic banter on these pages:

    Also, strangely missing from the article is that subscription journals also charge 'page fees', extra fees for colour figures, etc. which can be even higher than many open acess journals.

    1. Martin Eve

      Lecturer in English at University of Lincoln

      In reply to Chris Booker

      Hi Chris,

      Many thanks for the response (and same for all the above commenters).

      Just wanted to quickly add that it's very hard, when writing to a strict limit, to include everything. I did think about page fees but opted instead for a punchier head-on assault. You are, of course, absolutely correct though and more emphasis should be put on this kind of behaviour (as "author-pays") anyway in future!

      Best wishes,


    2. Chris Booker

      Research scientist

      In reply to Martin Eve

      Thanks for the reply Martin, actually just for record I was referring to the Science article not mentioning page fees (which I think really undermines the theme of that article that its all about OA publishers taking money), but realize now it looks like I was criticizing your article.

  4. Akbar Khan

    logged in via email

    Now we have real and proven data and result of the quality of SOME OA journals (neglecting the fact that it was not compared with similar subscription based journals and other weakness of this study). Even though this experiment is 'not perfect', but I am so happy to see that it throwing light on the quality of ‘screening and peer review service’ of OA journals. I strongly believe that the scholarly publishers should work like 'strict gate keeper' by arranging honest and sincere peer review service…

    Read more
  5. David Roos


    In my phone conversations with Bohannon, I indicated that my (admittedly anecdotal) experience suggests they would have obtained similar results with subscription-based journals in general, not just "the bottom tier" -- a phrase I would never use, and have no idea of how to define This error has been corrected in the on-line version.

  6. rory robertson

    logged in via email


    You appear to be complaining that journalist John Bohannon targeted only "open access model" journals and not also "subscription model" journals as well, because if he'd targeted subscription journals as well, his clownish paper would have been formally accepted for publication by heaps of them too. My guess is that, disturbingly, you probably are right.

    What is clear is that extraordinary faulty papers with false "findings" can be published very easily in formal "science" journals…

    Read more
  7. Ken Friedman

    University Distinguished Professor at Swinburne University of Technology

    John Bohannon's Nature article interested me greatly.

    Colleagues at Tongji University and I have been examining different publishing models, so I've followed the public response argument since the article appeared.

    It is not entirely clear from this article that Bohannon specifically does not investigate or criticise ALL Open Access publishing. He looked only at what is known as Open Access Gold, journals that charge publication fees for the articles they accept and publish. Where you get…

    Read more
    1. rory robertson

      logged in via email

      In reply to Ken Friedman

      Ken meant "Science" magazine rather than "Nature". Apart from that, I mostly agree with him on Bohannon's wonderfully revealing hoax.

      Beyond the nonsense-based papers regularly published as "peer-reviewed" science in pay-as-you-publish-anything-you-want e-mags - including MDPI's Nutrients "journal" - it would be interesting to know how many extraordinarily faulty papers in the past decade (say) have been formally (self) published while the influential lead author acted as the "Guest Editor" of the publishing journal:

      Does anyone have any post-publication peer-review stats on that?

  8. Bill Skinner

    Research Professor at University of South Australia

    It would seem to me that we need to ensure integrity and fairness in policy, before introducing something that can be misused and defrauded, and needing cleaning up after the fact.

    It seems that anyone can start an open access journal, if you're in the right country, and claim all sorts of volume of outputs. IT-savvy people can list a pile of journal papers from so-called open access journals that exist only on their own professionally-designed website, or list DOI's for papers that no one bothers…

    Read more