A much-vaunted scientific index measuring transformation has come in for a severe drubbing.
A scientific study known as the “equity index”, which centres on a mathematical formula that its authors claim provides an objective measure of the pace of university transformation, receives a scholarly slaughtering in the South African Journal of Science this week.
Coauthored by University of KwaZulu-Natal vice-chancellor Malegapuru Makgoba, an immunologist, and his colleague Kesh Govinder, a maths professor, the index purports to measure the pace of university transformation empirically and rank institutions accordingly (“University transformation will take 43 years, says study”, Mail & Guardian website, October 23 2013).
The equity index received enthusiastic political endorsement after its first publication in two pages of the same journal last year.
In a second and longer article, also published last year in the journal, the authors recorded this warm welcome and anticipated further political thrills: “It is remarkable that one simple index can positively impact two pivotal national policy documents [the education white paper and the Employment Equity Act].”
But three articles in the latest edition of the journal beg to differ. Two pronounce the death sentence on Makgoba and Govinder’s equity index on the grounds of what they argue is its fatally poor maths.
A third article finds life-threatening but treatable diseases such as anorexia of socioeducational insight compounded by suspected political aphasia.
Signs of life
This more positive diagnosis is Nico Cloete’s. The director of the Centre for Higher Education Transformation, and himself influential in the mid-1990s debates on post-school policy, he finds signs of life in the patient’s “serious attempt” to “reconcile the well-known … tension between equity and development”.
The attempt “is not entirely successful”, however. Anorexia becomes evident in the “fundamental flaw” of the equity index’s “assumption that the university should be a mirror of national demographics”.
“The university is a specific institution in society that is supposed to lead rather than reflect society,” Cloete writes. Makgoba and Govinder wonder whether slow progress in transformation is because of “passive resistance, denial, the abuse of autonomy or an abhorrence of accountability”, as Cloete summarises their speculations.
But, in what might be a diagnosis of politically aphasiac silence, Cloete writes: “Not once is the question raised [in the equity index study] as to the role of the national department of higher education and training and its contribution to the problem.”
It was Cloete and his colleagues who first revealed the problem captured in the now widely used acronym “neets” — those 18-to 24-year-olds “not in employment, education or training”. In 2007, they numbered 2.7-million, a Centre for Higher Education Transformation report showed in 2009.
A systemic problem
By 2011, they had grown to 3.2-million. That the government has done little about this “demonstrates fairly dramatically that the more serious problem is systemic rather than [as Makgoba and Govinder claim] institutional change”, Cloete’s current article argues.
These authors’ “assumption that the lack of transformation is simply the result of bad attitude is a common South African form of accusatory politics”.
On the maths of the equity study, Cloete leaves “further comment” to the other two articles. This they amply supply, with University of Cape Town statistician Tim Dunne’s title capturing the essence: Mathematical Errors, Smoke and Mirrors in Pursuit of an Illusion.
The “mathematical structure” of the formula the index uses is “well known”, Dunne writes. “However, its applicability to the setting described in the paper of Govinder and Makgoba is both logically incorrect for the intended purpose and morally dubious.”
He argues: “The danger of erroneous thinking rooted in a putative exclusive concern for moral purpose and social accountability is that any underlying logical or mathematical errors are too easily excused by the imputation of vested interest and mala fides to those who contest the dubious mathematics.
“Contrary voices can easily be caricatured as at least impervious or at worst opposed to the claimed moral purpose. Indeed, one of the hypotheses offered by the authors is that several universities (other than their own) are currently impervious to equity objectives.”
Negating the arguments
And in another article, UCT actuarial scientists Tom Moultrie and Rob Dorrington find “a number of algebraic, computational and conceptual errors [that] all but negate most of the arguments for and from the index”.
It is “quite surprising that a paper with so many flaws was deemed good enough to be published”.
The editor-in-chief of the journal, John Butler-Adam, said two reviewers — “a statistician and a higher education expert” — assessed the study in question. “The paper went through the review process twice … At no stage was there a breach of the journal’s required procedures,” Butler-Adam said. “It was of great concern to us that, without broader exposure … the views of the minister and the portfolio committee would be the only ones available and that they could, therefore, prevail, whether right or wrong.”
Last year Makgoba and Govinder presented the equity index in Powerpoint form to Higher Education Minister Blade Nzimande and the portfolio committee.
The presentation was also attached to a press release issued in October last year jointly by UKZN and the ministerial oversight committee on university transformation, which Makgoba chairs.
The equity index was a “seminal” study, the media release said. It would “drive” transformation in a way the second of the equity index articles described more fully: the “simple and objective” formula “punishes over-representation and under-representation [racially among staff and students, for instance], thus forcing organisations to properly plan their equity targets”.
Responding to the M&G’s detailed questions this week, Makgoba and Govinder wrote: “The [three] commentaries on our two articles only appeared in the journal on Monday. While we are engaging with the contents at the moment, it is far too soon for us to comment.
“Once we believe that a response is warranted, it will be submitted to the South African Journal of Science … We believe that science (and society) is better served by an academic, peer-reviewed debate in scientific journals before such discussions are debated in the media. A premature public discussion will serve only to confuse matters.
“Notwithstanding this, we stand by the equity index formula and its usefulness as a quantitative tool to measure the success of transformation.”