Hefce backs off citations in favour of peer review in REF
Research assessments in hard sciences will now be 'informed' by bibliometrics. Zoe Corbyn writes
The use of citations to determine the quality of academic work in the hard sciences is to be abandoned in favour of peer review in the new system being designed to replace the research assessment exercise.
However, information about the number of citations a scholar's work accrues could be provided to assessment panels to help "inform" their judgments in a range of subjects.
At a conference on the forthcoming Research Excellence Framework (REF) this week, the Higher Education Funding Council for England sketched out how it intends to assess the quality of research outputs in the system, which will determine the allocation of £1.6 billion of annual research funding from 2014.
"We just don't think bibliometrics are sufficiently mature at this stage to be used in a formulaic way or, indeed, to replace expert review," said Graeme Rosenberg, Hefce's REF project manager. "However, there is still scope for bibliometrics to inform the assessment process."
The announcement follows an interim report on the REF bibliometrics pilot exercise.
Although all decisions are yet to go to the Hefce board ahead of a consultation paper on the new system due later this year, the council told about 300 conference delegates that it was effectively sidelining the use of citations - a measurement of the number of times academics' work is cited by their peers - as the main determinant of research quality in the remaining hard science subjects for which it thought their use was possible. Instead, it is to draw up a list of subjects where it is appropriate for citations to inform peer review.
Dr Rosenberg also outlined Hefce's preferred model for assessing research quality. As in the RAE, institutions would continue to select which staff and which of up to four of their outputs are submitted.
The first exercise would cover work published between 2008 and 2012, and data on citations accrued for the papers in the same period would also go to panels.
"Panels might look at (citations for) each paper or (citations for) the whole submission. We are still working through the details," Dr Rosenberg told Times Higher Education.
Putting citation data for each paper before panels would be a radical departure from earlier plans, which until now have mooted citation profiles for institutions only in broad subject areas.
Charles Oppenheim, professor of information science at Loughborough University, attended the conference. He said it was now likely that pressure on scholars to increase their citations would grow.
"The new game to play is citations. One of the games, without any doubt, is that senior staff will encourage academics to cite within the institution," he said.
Dr Rosenberg also discussed how Hefce intended to assess the social and economic impact of research. The process would take place at a departmental level and would be based on a narrative backed by indicators and exemplars, he said.
The first REF would assess impact made between 2008 and 2012, he said, although the research itself could have been carried out earlier.