Retrospective impact assessment an ‘insurmountable’ task

Funding chiefs face “insurmountable” problems in their attempts to find a fair and effective way to assess the impact of research in the forthcoming research excellence framework.

This is the conclusion of the House of Commons’ Science and Technology Committee, which also warned today that any cuts to research funding in tomorrow’s Budget would have “potentially devastating consequences for science in the UK”.

As part of its inquiry into the impact of spending cuts on science and scientific research, the committee considered how both the UK funding councils and the research councils – both arms of the dual-support research-funding system – were approaching the vexed issue of implementing the government’s agenda to improve the impact of academic research.

In the proposals for the forthcoming REF, which will distribute more than £1.5 billion a year in research grants in England, the funding councils are looking at how to include a retrospective measure of the economic and social impact of departmental research.

As part of the same agenda, the research councils require grant applicants to set out what they think the future impact of their work may be.

The committee’s report says that the practical problems faced by the funding councils in trying to measure research impact retrospectively are “insurmountable”.

“We commend the lengths to which the Higher Education Funding Council for England has gone in order to consult and seek to meet the concerns of the academic community with regard to the inclusion of a retrospective assessment of impact within the REF, [but] we fear that their efforts may be in vain,” it says.

“It is our view that however meritorious the idea of awarding funding on the basis of past impact may or may not be, the difficulties associated with capturing past impacts effectively and allocating funds fairly on the basis of them will be insurmountable.”

On the research councils’ approach, it criticises their use of impact statements as “tie-breakers” in grant applications, but reserves the brunt of its criticism for their failure to communicate their approach adequately.

“If the research councils were not encouraging researchers to think about potential impact then it would be necessary for a select committee to recommend that they did,” it says.

“It seems that many assessors and those being assessed think that they are being asked to ‘predict’ impacts, when in fact the purpose is to stimulate thought about how impact may be developed…It is up to the research councils to improve the guidance they provide, and we urge them to act to clear up the misunderstanding.”

The decision of the MPs to back the research councils’ approach was criticised by Don Braben, visiting professor in the department of earth sciences at University College London, who has called for a “modest revolt” against impact statements.

“Why is the select committee determined to back the introduction of such an intensely controversial and to date undefined concept as impact?” he asked. “The last thing Charles H. Townes [who discovered the laser] would have wanted was to waste time considering what his discovery (if he actually made one) might be used for and who might use it…What the research councils should be doing is to create environments in which academic researchers can strive to achieve excellence.”

Another key message of the committee’s report was that any cut to research funding in this Budget or beyond would have dire economic consequences.

The pre-Budget report announced £600 million in cuts to the sector over the next three-year Comprehensive Spending Review period, but it did not say whether this would come from student support, higher education or science and research.

“We hope that the Budget will contain good news for science funding,” said Phil Willis, committee chair and Liberal Democrat MP for Harrogate and Knaresborough. “Anything less will have potentially devastating consequences for science in the UK.”

The report challenges the government to increase spending on science, saying it would if it were “truly committed to the principle of a knowledge-based economy”.

Nick Dusic, director of the Campaign for Science and Engineering, urged the chancellor to commit Labour to seeing through the Science and Innovation Investment Framework, which expires in 2014.

“All parties will need to set out strong science policy commitments before the general election if they are going to engage the science vote,” he said.

zoe.corbyn@tsleducation.com