International rankings don’t measure what matters

International rankings of universities are big business and big news. These systems order universities on the basis of a variety of criteria such as student to staff ratio, income from industry, and reputation as captured through public surveys.

Universities around the world use their rankings as marketing material and parents and prospective students make life choices on the basis of them.

But the methodology underpinning the Quacquarelli Symonds and Times Higher Education ranking systems and others like them would be unlikely to pass as a third year student’s research project. And yet high-status universities around the world spend time and money competing in this extravaganza rather than pointing out that the Emperor is wearing no clothes.

Why would they when the rankings reinforce their position as institutions of choice for those who can afford their fees?

As a researcher of higher education, I find it worrying that we’re held captive by these glitzy spectacles.


Imagine if a student indicated that their research project would be to develop a ranking of all universities. They would allocate 20% to whether current students and the general public thought the university was prestigious, 5% for the number of Nobel Prize winners on the institution’s staff, 30% for the number of research publications, and so on. Any academic advisor would throw the proposal out.

Some of these criteria are subjective. The weightings are arbitrary, important aspects of many universities are missing and the averaging of unrelated aspects to a final number is simply poor science which does not tell us much about the institution at all.

And yet this is exactly how rankings are determined.

The ranking method doesn’t add up

The methods behind the international university ranking systems vary but the underpinning methodology is identical. Convert proxy measures of a few academic activities into numeric metrics, add these together and come up with a ranking of institutions.

The criteria may be entirely unrelated to each other or may be poor proxies of the academic activity being measured. Reputation surveys and student throughput, for example, probably tell you more about how wealthy, and therefore selective, the university is than anything about the quality of their teaching.

Furthermore, the weighting of each criterion is almost entirely arbitrary. If publications and citations are worth 20% and web visibility is worth 10%, you will get one order of institutions. Change this to 25% and 5% and the entire list rearranges itself.

Many of the criteria could be seen as descriptive rather than evaluative. High pass rates and student:staff ratios are separate criteria in many systems, yet most academics would argue that they are measuring related issues.

Privilege begets privilege too, so universities that are the most prestigious and charge the highest fees can be the most selective in student enrolment and staff recruitment. Because higher education is not a meritocracy but largely reinforces social stratifications, these universities will then celebrate their inevitable success in getting students to complete their studies and graduate.

Many rankings focus on reputational rankings which ask employees, graduates, and the general public to indicate which institution is the best. But this becomes circular: a strong reputation leads to a strong reputation. This benefits well known universities and neither reflects nor benefits teaching and learning, research, community engagement, or any other academic activity.

When less wealthy universities attempt to compete, the cost can be a drain on resources that could be spent on activities more attuned to their context.

Many universities in South Africa are now chasing these rankings, even while admitting that they’re problematic. But participation is neither innocent nor harmless.

It’s not innocent because universities know the average citizen believes that these rankings say something about quality. This then influences choices as to where to study and work and whom to employ. Instead of using their research skills and academic integrity to pull these games apart, they expend a great deal of energy trying to improve their position.

Increasing research output in the global South is essential. If the rankings drive this process, well and good. But it’s harmful if the focus on publications and postgraduates comes at the expense of important factors that aren’t used in rankings.

What counts

Ranking systems don’t concern themselves with whether the university takes its community engagement responsibilities seriously. They don’t consider who students are and where they want to go. Ranking systems care only for the market – and higher education is a very large market.

Academics can and should speak out against these neocolonial processes that position all universities as striving to be identical and competing for market share.

But it’s difficult for universities to opt out of these games. The university where I work refuses to engage with ranking organisations. And yet it’s still included in these systems, as they draw on publicly available data.

Despite having among the highest undergraduate success rates and publication rates in South Africa, the lack of medicine and engineering programmes works against it. So too does its strong focus on community engagement and its small size – though these might be exactly why the university is a good fit for many.

The dodgy methodology is stacked against the institution but far more problematically, it’s stacked against most of the purposes set for higher education in South Africa’s white paper of 1997.

Nowhere do these systems concern themselves with transformation, social justice or the public good.

Sioux McKenna, Director of Centre for Postgraduate Studies, Rhodes University

This article is republished from The Conversation under a Creative Commons license. Read the original article.

Subscribe to the M&G

These are unprecedented times, and the role of media to tell and record the story of South Africa as it develops is more important than ever.

The Mail & Guardian is a proud news publisher with roots stretching back 35 years, and we’ve survived right from day one thanks to the support of readers who value fiercely independent journalism that is beholden to no-one. To help us continue for another 35 future years with the same proud values, please consider taking out a subscription.

Sioux McKenna
Sioux McKenna is the director of the Centre for Postgraduate Studies at Rhodes University

Related stories

A colossus with feet of clay

South Africa is disproportionately targeted by cybercriminals. Digital attacks call for digital solutions and technology is a the prime weapon in this fight

Journey through anxious Joburg

A new book has collected writing about the condition of living, yes, with a high crime rate, but also other, more pervasive existential urban stresses particular to the Global South

China’s resource-for-infrastructure deals

Are RFIs a viable model for aiding Africa’s economic development?

Football legend Maradona dies

The Argentinian icon died at his home on Wednesday, two weeks after having surgery on a blood clot in his brain

Covid vaccines: Hope balanced with caution

As Covid vaccines near the manufacturing stage, a look at two polio vaccines provides valuable historical insights

Under cover of Covid, Uganda targets LGBTQ+ shelter

Pandemic rules were used to justify a violent raid on a homeless shelter in Uganda, but a group of victims is pursuing a criminal case against the perpetrators
Advertising

Subscribers only

Covid-19 surges in the Eastern Cape

With people queuing for services, no water, lax enforcement of mask rules and plenty of partying, the virus is flourishing once again, and a quarter of the growth is in the Eastern Cape

Ace prepares ANC branches for battle

ANC secretary general Ace Magashule is ignoring party policy on corruption-charged officials and taking his battle to branch level, where his ‘slate capture’ strategy is expected to leave Ramaphosa on the ropes

More top stories

‘We struggle for water, but power stations and coal mines...

A proposed pipeline will bring water polluted with Gauteng’s sewage to the Waterberg in Limpopo to boost the coal industry during the climate crisis

Journey through anxious Joburg

A new book has collected writing about the condition of living, yes, with a high crime rate, but also other, more pervasive existential urban stresses particular to the Global South

Football legend Maradona dies

The Argentinian icon died at his home on Wednesday, two weeks after having surgery on a blood clot in his brain

Covid vaccines: Hope balanced with caution

As Covid vaccines near the manufacturing stage, a look at two polio vaccines provides valuable historical insights
Advertising

press releases

Loading latest Press Releases…