Impact may account for 30% of researchers' marks in REF

The impact of academics' work on the wider world could count for "at least 20 per cent" and "up to 30 per cent" of the assessment process under the forthcoming research excellence framework, one of its architects has revealed.

The weighting, which is likely to be highly controversial among academics, was mooted as a "starting point" for discussions with the sector by Graeme Rosenberg, REF project manager at the Higher Education Funding Council for England.

If adopted, it would represent a major shift in the way that academic work is assessed by Hefce as it distributes about £1.6 billion of quality- related research funding each year.

The research assessment exercise, which the REF is replacing, only implicitly took account of impact.

Speaking at a conference held by the Academy of Social Sciences in London last week, Mr Rosenberg said that increases in public investment in research over the past ten years had been made on the understanding that economic and social benefits would follow.

The system for assessing research was therefore being refocused "much more explicitly" on tangible effects.

"We strengthen our case for public investment in research if we can show that university research has actually produced a clear, visible impact and that the system itself recognises and rewards ... (it)," he said.

Mr Rosenberg added that although research outputs would remain the "main component" of the REF, impact and the rating of departments' "research environments" had to be given significant weighting to make their inclusion worthwhile.

"The starting point for a discussion on what is the right weighting for impact has got to be at least 20 per cent," he said.

Slides in his presentation suggested that it could be as high as 30 per cent.

Speaking to Times Higher Education after the event, he said: "If you don't give impact a big enough weighting, you have no influence on outcomes. We have got to show the Government that impact is an important part of the REF.

"The thinking on the weighting is that it must be substantial enough to make a difference to outcomes."

He stressed that the numbers should not be taken as fixed or even as an official proposal at this stage, adding that Hefce would publish one in the autumn.

Mr Rosenberg added that the funding council hoped to agree on a single number for the weighting across all subject areas, and confirmed that it would launch a pilot to test its plans for assessing impact in tandem with its autumn consultation.

Five-year outcomes

Under current proposals, impact would be measured over the period assessed by the REF from 2008 to 2012, although the research being measured may have been carried out earlier.

A department would delineate the impact of its work in a narrative supported by case studies - about one for every five to ten researchers submitted - alongside other relevant indicators.

Mr Rosenberg said impact would then be judged by panels of academics and users, and that grades would be based on its breadth and depth. The plan would be to produce impact profiles for each department.

However, academics at last week's conference expressed concern that the assessment could be based on which department told the best stories, and suggested that it was unfair to gauge retrospectively the impact of work done when the rules of the game were different.

Peter Main, director of education and science at the Institute of Physics, said that if impact accounted for 20 per cent to 30 per cent of the weightings, it would "massively" diverge from what the RAE's role had been - deciding which departments were producing the best-quality research and funding it accordingly.

Making a change of this magnitude "should not be done lightly", he added.

"It is understandable that the Government should be concerned about impact, but it is quite a different matter as to whether the REF should be (driving) that," Professor Main said. "It seems to me it should not be at all."

Nick Dusic, director of the Campaign for Science and Engineering, added: "Hefce should first consult on how it will assess impact before discussing the weighting given to it within the REF ...

"The pilot first needs to assess feasibility and ensure that the administrative burden is not greater than the gain."

zoe.corbyn@tsleducation.com

NEVER-ENDING STORIES: ESRC SEEKS INDEFINITE UPDATES

Researchers funded by the Economic and Social Research Council will be required to log the impact of their research projects - with pressure on them to do so in perpetuity.

The ESRC will introduce its latest final-reporting requirements for grant-holders from autumn this year, and other funding councils may follow its lead.

Under the arrangements, which have yet to be formally announced, ESRC researchers will be required to submit an "impact report" a year after the end of their award to show what their work has achieved for the economy and wider society.

Award-holders will then be encouraged - but not required - to keep such reports updated indefinitely by continuing to submit output and impact data.

The plan was set out by Phil Sooben, director of policy and administration at the ESRC, at the Academy of Social Sciences conference in London last week.