OECD may fill ‘missing link’ in world rankings

Learning outcomes project aims to get to grips with international teaching comparisons. John Morgan reports

An international study will measure and compare learning outcomes at universities around the world – and could provide the “missing link” for university rankings.

The Organisation for Economic Co-operation and Development (OECD) launched the Assessment of Higher Education Learning Outcomes (AHELO) initiative in Washington on January.

AHELO’s first phase will measure students’ “generic” skills, such as problem solving and critical thinking, through online tests adapted from the US’ Collegiate Learning Assessment.

The questions are intended to be non-specialised and answerable by undergraduates of all disciplines.

Fourteen nations are expected to take part in the full project, including the US, Finland, Italy, Japan and Mexico.

The goal is for all 30 OECD countries to participate eventually.

Richard Yelland, head of the education management and infrastructure division of the OECD’s Education Directorate, who is leading the initiative, said: “AHELO is a pioneering international attempt to assess the quality of higher education by focusing on what students have learnt during their studies and what skills they have acquired. Success will provide higher education systems and institutions with diagnostic tools for improvement that go far beyond anything currently available.”

Mr Yelland told The Chronicle of Higher Education: “This isn’t going to be a ranking. It is so much more. If we manage to produce reliable data, some people may well turn them into rankings, but that is not what this is about.”

For the 14 nations taking part in the full project, an average of 10 institutions per country will be involved, with about 200 students per institution taking part.

In the US, institutions in four states – Connecticut, Massachusetts, Missouri and Pennsylvania – are participating.

Ben Wildavsky, author of a forthcoming book on the globalisation of higher education, The Great Brain Race, described AHELO as “a very promising initiative”.

He added: “It has the potential to provide the missing link in existing university assessments and rankings, which have often been faulted for overemphasising research, neglecting undergraduate education and focusing on input measures at the expense of rigorous measures of student-learning outcomes.”

Mr Wildavsky, senior fellow in research and policy at the Ewing Marion Kauffman Foundation in Kansas City, a body that promotes entrepreneurship, added: “If the OECD can pull it off, it will have filled a major gap and moved us towards a world in which universities have much better incentives to improve an important part of their work – teaching students.”

Mr Wildavsky said the involvement of the US, with support from its Department of Education, was key.

However, there will be “technical barriers to coming up with truly valid, reliable and comparable tests across nations”, he added.

“I also feel that the best measures of university effectiveness would include not only student learning but also research prowess – but perhaps that will come later if and when AHELO proves its worth,” he said.

Phil Baty, editor of the Times Higher Education World University Rankings, said: “One of the biggest weaknesses in the current global rankings systems is that there is nothing that can fairly measure teaching quality across different countries and cultures, so the AHELO project is being watched very closely indeed. But inevitably it will be very difficult and will take a long time before there are any serious data for detailed comparisons between institutions.

“In the meantime, Times Higher Education is working closely with its new rankings partner, Thomson Reuters, to make sure that there are more sensible and rigorous measures of teaching quality for the World University Rankings in 2010 and beyond. We are working on a far more sophisticated reputational survey of world academics, where we expect teaching quality to be unpicked in a way that’s never been done before in world rankings.”

john.morgan@tsleducation.com

Already registered?

Sign in now if you are already registered or a current subscriber. Or subscribe for unrestricted access to our digital editions and iPad and iPhone app.

Register to continue  

You've enjoyed reading five THE articles this month. Register now to get five more, or subscribe for unrestricted access.

Most Commented

  • Man measuring bar graphs with tape measure

An Elsevier analysis explores the viability of a ‘smarter and cheaper’ model

  • David Willetts

The former universities minister discusses the reforms that reshaped higher education and his first steps into academia

  • Man holding a box filled with work-related items

Refusal by John Allen to obey instruction from manager at Queen Mary University of London led to his sacking, tribunal rules

  • A black and white crowd scene with a few people highlighted

What are the key issues local union branches are dealing with, and how do they manage relationships with institutions in what many activists argue is an increasingly confrontational environment?

  • Muslim woman at graduation ceremony, Barbican, London

Sector called on to embrace faith-related concerns in intellectual debates