REF could penalise those working across disciplines

2 October 2008

The system being set up to replace the research assessment exercise could discourage academics from working with colleagues outside their own discipline even though the Government is trying to encourage interdisciplinary work, new research suggests.

Under the forthcoming research excellence framework, which will replace the RAE after this year, the quality of an academic's research in science subjects will be judged in part on the basis of how many times their published research is cited by their peers.

But a study conducted by researchers at the School of Computing and Information Technology at the University of Wolverhampton has found that interdisciplinary and multidisciplinary research is less likely to be cited than monodisciplinary research - in some subjects as much as four times less.

Mike Thelwall supervised the study, the results of which are published in this month's Journal of the American Society for Information Science and Technology in the paper "Is multidisciplinary research more highly cited? A macro-level study". He said the findings threw "a spanner in the works" over the Government's plans to develop the REF.

"Governments around the world are trying to promote interdisciplinary and multidisciplinary research. The UK Government is moving to the REF and a reliance on citations ... (But) doing interdisciplinary and multidisciplinary research is going to get you less recognition and funding under the REF. Something the Government wants to promote is going to be discouraged," he said.

The study used standard categories from citation databases to divide science and social science journals into those that cover a single subject and those that cover multiple subjects. It then looked at about 960,000 papers published in the journals in 1995 and their subsequent citations.

Although there are differences between subjects, the overall results show that papers published in single-subject journals receive more citations than those published in multiple-subject journals.

The trend was most pronounced in the life sciences, the health sciences and the physical sciences. In these subjects, "on average, the level of citation for monodisciplinary (articles) was more than double that for multidisciplinary articles," the study concludes.

Subjects faring particularly badly include astronomy, physics and chemistry. Papers in astronomy and physics in multidisciplinary journals received 4.2 times fewer citations than those in single-subject journals, and chemistry papers in multidisciplinary journals received 3.7 times fewer citations.

The results follow the Higher Education Funding Council for England's publication of two studies by the University of Leiden that were commissioned to feed into the development of the REF pilot exercise for citations.

One report, Appraisal of Citation Data Sources, found that the Scopus citation database should be considered as a "genuine alternative" to the Web of Science database, which was originally proposed as the REF's source of citation data.

A spokesperson for Elsevier, the company that operates Scopus, said the conclusion of the Leiden report was "very pleasing" and "objectively demonstrated" the ability of Scopus to support the REF.

The other report, Development of Bibliometric Indicators of Research Quality, found that the parameters used to construct the citations indicator could have a significant effect on the outcome, and recommended further work.

zoe.corbyn@tsleducation.com.