Imagine that a newspaper decided to create a table ranking the world’s cities. Is Moscow better than Sydney? Would Hong Kong squeeze in above Manchester? Or Bangkok above Brighton?
It would be a nonsensical exercise. Better in what respect? They are all vastly different types of human settlement, meeting different aspects of human need in different cultures and climates. True, there would be no shortage of data, though it would be of variable robustness and gathered on different bases in different countries. We could compare expenditure on public transport, or the area of open space, or unemployment. But to come to a single ranking, we would then have to manipulate the data: should a measure of the state of public health outweigh a measure of investment in public transport?
Exactly the same problems arise with world rankings of universities. Each time a new set is published, universities shoot up or down the league tables, and vice-chancellors the world over engage in their traditional exercise of deploy (if up) or deplore (otherwise).
But, in the real world, nothing has changed. The relative strengths of United Kingdom universities vis-Ã -vis their international peers vary only marginally and incrementally year on year. All that has changed is that the league-table compilers have adopted new metrics and applied new weightings to them, but they still fall miles short of capturing anything like the variety, the dynamism and the diversity of the modern university.
The basic problem is that there is no definition of the ideal university. Is it big or small? Is it better at research or at teaching? Is it a specialist science and technology institution with heavy research income, or does it focus primarily on the humanities and social sciences? Is it adventurous or risk averse?
Next, what are the data? On teaching, there are simply no robust data upon which global comparison can be made. Some tables rely on curious proxies, such as the number of PhDs awarded for each academic, the number of undergraduates admitted for each academic and the income raised by each academic. None of these say anything about the quality of teaching.
We might expect measures of research excellence to be robust, because top research is a globalised activity. Yet we would be disappointed. Take research income. There is a world of difference between scientific and medical institutions — where research is expensive and grants tend to be large — and other types of institution. This helps to explain why a world-class institution such as the London School of Economics can be ranked in one table no higher than 86th.
What’s more, research income is an input measure and will tend to be higher in countries investing seriously in research. It does not speak to the quality of output, where UK universities have consistently demonstrated an outstanding return on investment.
Research citations can be a measure of quality, but need to be intelligently applied. There are big cultural variations between disciplines, which citation indices are not well attuned to capturing.
This isn’t easy territory. Get it wrong and you end up with some truly bizarre results. For example, one of the recent sets of rankings concludes that Egypt’s Alexandria University trumps both Stanford and Harvard for its influential research measured by impact.
Global rankings have afforded annual light entertainment, but they are now overreaching themselves. They do a disservice if they influence student choice, or come to be treated as a performance measure by the leaders of hugely diverse institutions. — Guardian News & Media 2010
Malcolm Grant is president and provost of University College London, which in current world rankings is variously placed fourth (QS), 13th (Global Universities Ranking), 17th (Taiwan index), 21st (Shanghai Jiao Tong), 22nd (Times Higher Education) or 81st (4iCU)