Research intelligence - Not so good with numbers

Statistics crop up in many different subjects from the sciences to the social sciences, but are scholars guilty of abusing them rather than using them accurately?

Questions have been raised about academics' use of statistics by the findings of an independent review into the science of the "Climategate" affair. The review, led by Lord Oxburgh, cleared scientists at the University of East Anglia's Climatic Research Unit (CRU) of allegations that they manipulated data and tailored research results to support an agenda predicated on the existence of man-made climate change.

But the findings, published last month, also suggest that the scientists could have used better statistical techniques to analyse their results.

"We cannot help remarking that it is very surprising that research in an area that depends so heavily on statistical methods has not been carried out in close collaboration with professional statisticians," the report notes.

"Indeed, there would be mutual benefit if there were closer collaboration and interaction between CRU and a much wider scientific group."

The issue identified in the CRU's work was not one of statistical techniques being used inappropriately but rather that those used were not always "best for the purpose".

However, Denis Mollison, emeritus professor of applied probability at Heriot-Watt University, said that the problem of poor-quality statistical analysis was common elsewhere in the academy.

"There are lots of fields of science where statistics crop up from time to time, (but academics) don't have statistical analysts to help or don't even realise that they have a problem," he explained.

The scientific literature was being affected as a result, Professor Mollison said.

"There are lots of papers where academics have done their best, but while their science is good, their statistics are rubbish. They are not aware of it, the peer reviewers are not aware of it, and journal editors may not realise that (the papers) need to be reviewed by proper statisticians."

Part of the problem, he said, was that many scholars relied on courses they had done decades ago. Another issue was that there were not enough well-trained statisticians to go round.

No points for guessing

Social science in particular is an area that could benefit from a better and wider understanding of statistical techniques, according to Sheila Bird, senior statistician at the Medical Research Council's Biostatistics Unit and visiting professor at the University of Strathclyde.

She said that while appropriate experimental design and statistical methods were de rigueur in medical fields, where the issue has been high on the agenda since the 1970s, the same could not always be said about the social sciences.

"There is a lack of randomised controlled trials, particularly in the social sciences," said Professor Bird, who is chair of the surveys, design and statistics subcommittee of the Home Office Science Advisory Committee.

"We need much better designed experiments and a much greater appreciation of the need for precision in estimation and statistical reporting."

Her comments chime with the findings of a recent study funded by the Economic and Social Research Council.

The International Benchmarking Review of UK Sociology, published last month, says that UK sociologists make "relatively little use of statistical methods". In fact, it continues, the discipline can be "skeptical ... or even hostile" to quantitative methods.

"Most British-trained sociologists cannot read the quantitative literature in sociology with any degree of understanding," the report says, noting that this "deficit" can be traced to the absence of statistical techniques in undergraduate sociology programmes.

David Hand, professor of statistics at Imperial College London and a member of the Oxburgh review, agreed that the use of statistics in the academy was a matter of concern.

"It is a good idea (for a researcher) to get a statistician, and it is a good idea to get one before you collect your data," he said.

But others think that this is not always practical.

Anthony Atkinson, a professor in the department of statistics at the London School of Economics, said that statisticians were very often occupied with their own work.

Like other academics, they are employed to teach and publish research, and "are not employed to be helpful any more than members of other departments".

zoe.corbyn@tsleducation.com.