Cookie policy: This site uses cookies to simplify and improve your usage and experience of this website. Cookies are small text files stored on the device you are using to access this website. For more information on how we use and manage cookies please take a look at our privacy and cookie policies. Your privacy is important to us and our policy is to neither share nor sell your personal information to any external organisation or party; nor to use behavioural analysis for advertising to you.

RAE reviewers accused of going over the top on scores

Concerns about 'grade inflation' as panels 'put too many people in the 4* box'. Zoe Corbyn reports

Concerns have been raised that reviewers for the research assessment exercise have been "too generous" in awarding the top grade.

The results from the 2008 RAE, published on 18 December, said that 17 per cent of all research submitted by more than 50,000 researchers was "world leading" (4*). Some 150 of the 159 institutions included in the exercise were shown to have at least 5 per cent of world-class activity in at least one department.

Initial celebration of the results as a clear sign of the UK's international research excellence has been replaced by complaints about "grade inflation" and warnings that this could affect the distribution of funding.

Observers have expressed concerns about what they say are the "very high" percentages of 4* work in certain subject areas. Some peer-review panels in arts-based subjects, as well as in business and economics, seem to have been more generous in awarding the top grade than those marking sciences or languages.

In communication, culture and media studies, for example, the highest-scoring department saw 65 per cent of its research activity graded 4*. Dance and music have similarly high figures.

However, the top-scoring physics department saw just 25 per cent of its research graded 4*.

Top departments in bioscience-related subjects, where the UK is considered to excel, hover between 30 and 40 per cent for 4* research activity. In languages, the departments with the highest percentages of 4* research activity are grouped in a similar bracket.

"It would have been better if all the subjects had used the top category (4*) less," said Andrew Oswald, professor of economics at the University of Warwick. "The whole point of the categorisation from 1* to 4* was to try to separate out work that was on the edge of winning Nobel prizes, and to get people to understand the difference between good work and absolutely breathtaking, iconoclastic work. I think that a general problem with the RAE is that all panels have put too many people into that 4* box."

Eric Thomas, vice-chancellor of the University of Bristol, said that although the 17 per cent figure for 4* overall was "probably" about right, he was concerned that several panels had been too generous in their interpretation of 4*.

"Some results show more than 50 per cent 4* in some institutions in particular units of assessment," he said. "The grade 4* was deliberately described as world-leading research which I would have thought of as a minority, not a majority," he said.

Departments were graded by 67 subject subpanels moderated by 15 main panels, which also met to ensure consistency in how the grades (of 4* for world-leading research through to 1* for nationally recognised research) were interpreted.

The Higher Education Funding Council for England, which ran the RAE, has said that the results across panels are supposed to be "broadly comparable".

But one panel member, who asked not to be named, agreed that it had not been "too hard" to get a 4*.

"Is it the Nobel prizewinning paper or something that is much more common but nevertheless drives the field forward? ... In the end we decided that it was something that happened to good people every year," he said.

• The RAE's exposure of pockets of research excellence across a range of teaching-intensive institutions could lead to extensive poaching of staff by the more powerful research institutions, the sector was warned this week.

This RAE provided, for the first time, a "research profile" for each department rather than a single summative judgment. The profiles highlighted pockets of excellence in departments that, overall, are not world class.

One senior university executive commented: "It is the other side of the exercise. You identify the very good universities so people want to be in them, but ... any identification of clever people means the market will (naturally) look to them. When we are looking to hire at whatever level, we want someone good so we look around to see where they are."

zoe.corbyn@tsleducation.com.

Readers' comments (1)

  • Alex's solution neatly encapsulates the emerging role of universities in bolstering a failing market economy.

    (A good point - how would one recognise international excellence in such a subjective and self-serving field as "media studies"? The simplest solution is for HEFCE to withdraw all funding from such areas and concentrate on the STEM subjects on which our economic future depends.)

    So that academia could be fully market-serving, I presume? Come to that why not close down all departments not fixated on industry?

    Unsuitable or offensive? Report this comment

  • Print
  • Share
  • Save
  • Print
  • Share
  • Save
Jobs