Keep peer input in REF, urge panels

The new system being set up to determine the allocation of £1.5 billion a year in research funding must maintain peer review at its core.

This is the message from a number of the expert review panels judging the work of more than 50,000 academics under the 2008 research assessment exercise - the final RAE.

The RAE is to be replaced by the research excellence framework, which will judge research quality based on numerical indicators, such as the number of times an academic's work is cited by others. A reduced form of peer review is due to be maintained.

This week, in their subject overview reports, some panels warn against the overuse of "metrics", or numerical indicators.

The computer science panel says that "citation counts were poorly correlated with the panel's assessment of the work examined". World-leading theoretical work, for example, had "low citation counts".

The panel for law says it came to the "unanimous" view that "detailed peer review of outputs is the only method that will attract the confidence of our discipline".

The sociology panel says that peer review is crucial as "world-leading articles were found in the full range of publication outputs".

The panel reviewing languages research says: "Our efforts have shown not only that work of genuine world-leading stature is to be found in unrefereed outlets, but that articles in well-respected international journals do not always meet the highest standards."

The education panel says peer review has been "essential" in making fair judgments.

Other panels also chose to use their official RAE reports as a platform to defend peer review against arguments that numerical indicators provide a more objective measure of quality. The history, Classics and philosophy panel says: "It is not accurate to characterise peer review as in some sense dangerously subjective.

"In fact, we were more aware of the difficulties of interpretation of supposedly more objective data ...".

The panel reviewing economics, business and management research says: "Although much top-quality work was indeed published in what are generally regarded as leading journals, top-quality work could also be found in journals occupying a lower position in conventional rankings.

"Similarly, some of the work considered that had been published in so-called leading journals was thought to be of less than top quality ... There was also a considerable amount of work published in books or other formats, some of which was of world-leading quality."

john.gill@tsleducation.com.