Advertisement
Advertisement
Advertisement
Leader: Put all the results out in the open
12 November 2009
Researchers, government and society benefit when research is made freely available, so the sooner it is mandated, the better
Impact is certainly having an impact and could be measured in the amount of hot air generated. Looking at it dispassionately, one can see both sides of an argument that is becoming increasingly polarised. Of course academics want to have impact; no one wants their work ignored. And in a chilly fiscal climate, the Government needs comforting facts and figures in which to wrap its spending plans to satisfy an exasperated electorate.
But the concept has been ill-defined and the message to academics clumsy, appearing to tell them that they have to be both producers of research and salesmen for its impact; the latter a particularly difficult task given that achieving impact is mostly outside their sphere of influence. As a poster on our website asks: is the business of the academic as much about dissemination as it is about discovery?
If it is, the Government is certainly missing a trick with open access. The research excellence framework has been designed with impact in mind, proposing that 25 per cent of funding is allocated according to the economic and social benefits of submitted research. If the Higher Education Funding Council for England really wants to recognise research that can "deliver demonstrable benefits to the economy, society, public policy, culture and quality of life", instead of trying to force researchers to demonstrate this, it should put a requirement into the REF that all research papers submitted be mandated open access.
It's obvious that if we want policymakers and small businesses to draw on the fruits of the academy's labours, we have to set out the stall so that they can see them. Both these sets of "research users" lack the logins to the subscription-based journals publishing the papers and the money to pay to access them. Is it right that academic research paid for by the taxpayer remains so unavailable to the people who are likely to make the most use of it?
But it is not only common sense that recognises open access - the subject of our cover feature - as a powerful tool to deliver impact. A recent pre-print by Stevan Harnad et al of research into citation impact shows that authors whose papers are made open access are cited significantly more than authors whose articles are available only to subscribers. The Open Citation Project provides ample evidence of this (http://opcit.eprints.org/oacitation-biblio.html).
When the House of Commons Science and Technology Committee highlighted open access in its 2004 report Scientific Publications: Free for All?, it favoured a change in the publishing model to open access and was critical of the Government for giving the issue so little attention. "All UK higher education institutions (should) establish institutional repositories on which their published output can be stored and from which it can be read, free of charge, online, (and) research councils and other government funders (should) mandate their funded researchers to deposit a copy of all of their articles in this way," it said.
Although most of the recommendations fell victim to intense lobbying from subscription publishers, there have since been moves in the right direction. The research councils have all introduced policies mandating that papers deriving from the work that they publicly fund be made available in open-access repositories. All that remains now is for that other arm of dual support to be twisted to do the same.
ann.mroz@tsleducation.com.
Readers' comments
It's unbelievable that this complicated issue can be easily solved. It's hard even to estimate the need for a quick solution. But, I believe that if start is given by some institutions, the others will have to follow. What will remain, however, is for governments to give substantial sums to publishers in order that past publications be given open access. Scientific knowledge should not be divided into two parts, pre-open and post-open access.
It’s not “lobbying from subscription publishers” that has stalled open access, it’s the realization that the simplistic arguments of the open access lobby don’t hold water in the real world. That’s not helped by the open access lobbyists constantly referring to the same biased and dubious ‘evidence’ (much of it not in the peer reviewed literature). Take for example the comment on citations above. Phil Davis, former librarian turned academic at Cornell, and colleagues published the first randomized trail of open access, downloads and citations in the BMJ (Davis, P.M., Lewenstein, B. V., Simon, D. H., Booth, J. G. and Connolly, M. J. L. Open access publishing, article downloads, and citations: randomised controlled trial BMJ, 2008;337:a568, published 31 July 2008). The findings? “Open access articles were no more likely to be cited than subscription access articles in the first year after publication”
Ian Russell has unfortunately neglected to mention the fact that the paper he cited by Phil Davis, published in the BMJ, was an interim result in a trial intended to run for 4 years. Admittedly the paper by Phil Davis also omitted to mention that extremely important point, leading to an apology by Trish Groves, Deputy editor of the BMJ, for allowing an interim result to be published without making it clear that it was an interim result. Had Ian Russell actually read the report and the subsequent correspondence, available to all on the BMJ website*, he might have realised that Open Access has more merits than he anticipated... http://www.bmj.com/cgi/eletters/337/jul31_1/a568
It's especially significant that Ian Russell -- CEO of the Association of Learned and Professional Society Publishers (which, make no mistake about it, includes all the big STM commercials too) -- should be saying "It’s not 'lobbying from subscription publishers' that has stalled open access, it’s the realization that the simplistic arguments of the open access lobby don’t hold water in the real world... [with] open access lobbyists constantly referring to the same biased and dubious ‘evidence’ (much of it not in the peer reviewed literature). Please stay tuned for more peer-reviewed evidence on this, but for now note only that the study Ian Russell selectively singles out as not biased or dubious, the "first randomized trial" (Davis et al 2008), which found that “Open access [OA] articles were no more likely to be cited than subscription access articles in the first year after publication” is the study that argued that in the host of *other* peer-reviewed studies that *have* kept finding OA articles to be more likely to be cited (the effect usually becoming statistically significant not during but after the first year), the OA advantage was simply a result of a self-selection bias on the part of authors: They selectively made their better (hence more citeable) articles OA. Russell selectively cites only this negative study, whose result is more congenial to the publishing lobby, and selectively ignores all the positive (peer-reviewed) studies, as well as the critiques of the study in question (as being based on too short a time interval and too small a sample, not even replicating the effect it was attempting to demonstrate as being merely an artifact of a self-selection bias). Russell also selectively omits to mention that even the Davis et al study found an OA advantage for downloads within the first year -- with other peer-reviewed studies having found that a download advantage in the first year translates into a citation advantage in the second year (Brody et al 2006). But fair enough. We've now tested whether the self-selected OA advantage is reduced or eliminated when the OA is mandated rather than self-selective. The results will be announced as soon as they have gone through peer review. Meanwhile, place your bets... -- -- Brody, T., Harnad, S. and Carr, L. (2006) Earlier Web Usage Statistics as Predictors of Later Citation Impact. Journal of the American Association for Information Science and Technology (JASIST) 57(8) pp. 1060-1072. http://eprints.ecs.soton.ac.uk/10713/ -- -- Davis, PN, Lewenstein, BV, Simon, DH, Booth, JG, & Connolly, MJL (2008) Open access publishing, article downloads, and citations: randomised controlled trial British Medical Journal 337: a568 http://www.bmj.com/cgi/content/full/337/jul31_1/a568 -- -- Harnad, S. (2008) Davis et al's 1-year Study of Self-Selection Bias: No Self-Archiving Control, No OA Effect, No Conclusion. http://openaccess.eprints.org/index.php?/archives/441-guid.html -- -- Hitchcock, S. (2009) The effect of open access and downloads ('hits') on citation impact: a bibliography of studies. http://opcit.eprints.org/oacitation-biblio.html
When reaserch has been funded by the tax payer, and the researcher has had adequate time to analyse and publish the work there is a very proactive way of licensing by using the appropriate Creative Commons (CC) licensing. In Australia the Australian Bureau of statistics uses a Creative Commons licence on publicly avialiable reports and is clearly available on the website with the material. This movement of open access (when applied appropriately) to summary findings alow the public, nGOs and businesses to have access to data and findings. There is a great website on the Government Information Licencing Framework (GILF) where information is availabe on how to assess an information product to apply CC licensing.
Citation and impact are not easy to quantify as different studies have shown and therefore should not form the basis for arguing in favour of open access. Intuitively if an article is made open access then it will have higher visibility and gain greater citation - but this is not necessarily true. Studies have shown variable citation behaviour in which the access of an article appears to have no bearing. For example higher citation of the same article within different (higher "Impact Factor") journals (Vincent Larivière and Yves Gingras on http://arxiv.org/abs/0908.3177), and the "cluster-effect" of citations whereby authors follow citation trails laid by papers that they read resulting in a reduction in the number of articles being cited (James Evans in Science, 18 July 2008). I guess (as with all statistics) it is quite possible to find a study that supports one's point of view. <br>I agree with Ian Russell that accusing publishers of "intensive lobbying" is inflammatory since both sides have formed lobbying bodies (in fact I perceive the OA lobby to be the more vociferous one). Many publishers (commercial or not) are offering authors the opportunity to publish OA within their journals. The current problem is that someone has to pay for the operation of scholarly communication, and there is no simplistic answer that will provide an overarching solution for all disciplines in all parts of the world - as much as both publishers and other lobbyists would like there to be.
Citations (and downloads) are countable, and are counted. Research usage and impact is not *synonymous* with citations or downloads, but citations and downloads are certainly *measures* of research usage and impact. If OA generates more citations (and downloads), that is most definitely a compelling basis for arguing in favor of OA. (Indeed, no argument for OA could be more compelling than that OA increases research impact; certainly not the argument that OA makes journals more affordable, nor the argument that it makes accessible to the general public peer-reviewed research journal articles most of which the general public has no interest in reading [nor do most peers!]. So every download and citation matters for this esoteric content, written by specialists to be read, used, applied and built-upon by specialists, for the sake of research progress, and thereby for the benefit of the general public.) # # # # No one has said it is *necessarily* true that OA articles are used and cited more. The empirical research on the OA impact advantage and the "self-selection bias" is being conducted to see whether it is true as a matter of empirical evidence. # # # # (1) Not every OA article will be cited more: only the ones that are found useful enough to be citeable will. And the more useful an article is, the greater the observed OA citation advantage. That is why the empirical question about causality is: "Are OA articles more likely to be cited because they are OA? Or are they more likely to be OA if they are more cited (the self-selection bias)? (2) 80% of citations are citations of the top 20% of articles. (3) The top journals are both more likely to publish the top articles and more likely to be cited. (4) Gingras & Larivière are our co-authors on the study testing mandated OA against self-selected OA that I mentioned (and that is being submitted for peer review). # # # # And on the "cluster effect," see: Larivière, V; Gingras, Y; & Archambault, E. (2008) The decline in the concentration of citations, 1900-2007 ["This paper challenges recent research (Evans, 2008) reporting that the concentration of cited scientific literature increases with the online availability of articles and journals"] http://arxiv.org/pdf/0809.5250 # # # # (It is indeed always "quite possible to find a study that supports one's point of view" -- and that's called a self-selection bias. The remedy is properly controlled studies and meta-analyses to determine where the preponderance of the evidence lies (in the metaphoric, not the mendacious sense!). # # # # (The big difference between the pro- and anti-OA lobbyists is that the former are not doing it for the money!) Some publishers are indeed offering authors the option of OA -- in exchange for paying the publisher a hefty price for it! But what is under scrutiny here (the self-selection-bias hypothesis) is not this generous offer on the part of some "Open Choice" hybrid-Gold publishers, but the alternative, which is author Green OA self-archiving, and whether that enhances citations, or is merely a self-selective bias toward self-archiving the top articles. # # # # "Scholarly communication" is being paid for, handsomely, today, by institutional journal subscriptions. So that is definitely not the "current problem." The problem is that not all the intended users for which this research is being conducted can access it, because their institutions can only afford to subscribe to a small fraction of the peer-reviewed journal corpus. That's the "current problem." And the -- yes, simple -- solution is for researchers' institutions and funders to mandate that all their own journal article output must be made freely accessible online -- to all its intended users (not just to the ones at the institutions that have subscriptions to the journal in which it happens to be published) -- by ensuring that all authors self-archive their refereed final drafts in their own institutional repositories immediately upon acceptance for publication. There are no disciplinary or geographic differences for the peer-reviewed journal article corpus in this overarching solution -- neither in its benefits nor in its feasibility -- much though some publishing lobbyists might wish there were.
Actuallly, I am aware of the post publication comments on the BMJ article by Davis et al mentioned by Stevie Gamble, I do encourage people to read both the paper and the comments. Trish Groves states that the fact this was an interim analysis should have been "clearer in the paper" but describing this as an apology is rather misleading and could be seen as seeking to discredit the paper. In anycase (i) the fact it was an interim analysis does not invalidate the findings and (ii) the further studies of the authors since the publication of this initial article support the findings.
Ian Russell continues to be economical with the actualité; the study is still in progress and the subsequent findings he refers to are also interim results. As for discrediting the paper, the fact that the author chose to misrepresent by omission the nature of his research speaks for itself, as does the fact that Trisha Grove's apology was needed because she "was involved in the peer review of this study by Davis et al". It is, after all, axiomatic that any competent peer review would have immediately identified the fact that the project was supposed to cover a four year period...
There is some confusion here on the timely reporting of our research results: Indeed, I am working on a project that has been funded by the Mellon Foundation for a total of four years. This part is true. However, no one (other than Harnad and Gamble) ever announced that I would wait until the conclusion of the grant to begin reporting the results. If there is any apology that needs to be made, it is by Stevan Harnad, who continually uses this piece of misinformation to discredit the study and the intentions of its authors. It is also remarkable that he finds this argument so compelling.
Whilst contemplating the need for apologies, the reader might care to take a peek at the following methodological critiques of Phil Davis's studies to date: Davis et al's 1-year Study of Self-Selection Bias: No Self-Archiving Control, No OA Effect, No Conclusion http://openaccess.eprints.org/index.php?/archives/441-guid.html Confirmation Bias and the Open Access Advantage: Some Methodological Suggestions for Davis's Citation Study http://openaccess.eprints.org/index.php?/archives/451-guid.html On Self-Selection Bias In Publisher Anti-Open-Access Lobbying http://openaccess.eprints.org/index.php?/archives/655-guid.html