Quality evaluations

How much do student evaluations of the quality of academic staff in universities reflect the quality of academic research? The data from the Times Higher Education Student Experience Survey 2015 published in THE last week show that staff quality evaluations are a fairly good measure of research quality.

If we correlate the grade point average scores from the research excellence framework with the quality of staff measure in the student survey, there is a moderately strong relationship (+0.46) for 91 higher education institutions in England. This means that good researchers tend to do well as teachers, as the REF and the students are by and large in agreement with each other. However, as is well known, the GPA figures were doctored by excluding significant numbers of staff from the exercise. This prompted THE to produce intensity-adjusted GPA scores to compensate.

If we correlate the intensity-weighted GPA with evaluations of the quality of staff in the student survey, the relationship is much stronger (+0.59). This means that there are some really good teachers who are not necessarily strong researchers, and they contribute to the overall quality of the staff. It should be no surprise that the students are evaluating all the staff, not just the ones included in the REF.

But how does this all translate into changes in institutional funding made by the Higher Education Funding Council for England? The correlation between funding changes between 2014 and 2015 and the quality of staff measure in the student survey is zero (-0.03), and the same is true for funding changes and overall student satisfaction ratings (-0.01), indicating that whatever Hefce is incentivising, it is not the quality of staff in universities.

Paul Whiteley
Professor of government
University of Essex