/ 25 February 2011

CHE audits ‘might never be the same’

Che Audits 'might Never Be The Same'

The Council on Higher Education’s (CHE) institutional audit of the University of KwaZulu-Natal (UKZN) has had considerable coverage in the Mail & Guardian in recent weeks. UKZN has clearly and emphatically stated that it did not suppress any audit report. In fact UKZN did not receive a report to suppress.

As the CHE made explicit in its response in the M&G, the university complied with all the requirements of the CHE institutional audit process. It produced a comprehensive self-evaluation report working through much of 2007 and 2008, coming on the heels of a major and challenging merger of two very different universities — the only university merger to involve a historically advantaged white English university and a disadvantaged black university.

Single schools and single faculties were created in the merged UKZN. Whole faculties with staff and students were moved across campuses, academic programmes were integrated while their accreditation was maintained and the university continued to improve its research productivity. UKZN is the only merged university that features among the top research universities in the country.

As in all institutional audits, a panel was constituted by the CHE and agreed to by the university. Having signed confidentiality agreements with the CHE, the panel arrived in October 2008 and was given unfettered access to the university at large and all kinds of documents.

The UKZN community listened, without recording or responding, to the oral feedback by video streaming at the end of an intensive week of interviews in the same month. UKZN then waited for the draft report. What it got instead was its name splashed across a newspaper headline with a leaked letter authored by the chair of the audit panel, Professor Martin Hall (The Mercury, January 23 2009).

What is unprecedented in the seven years since the CHE institutional audits started are the flawed processes that have dogged this audit since the oral feedback.

It is important to be clear about the mandate of the higher education quality committee (HEQC) of the CHE during institutional audits and the brief of its panel members: to audit the quality assurance mechanisms of higher education institutions and their “quality management arrangements”.
The CHE “Criteria for Institutional Audits” (June 2004) are explicit about this: “The HEQC audit will focus on an institution’s policies, systems, strategies for quality management of the core functions of teaching and learning, research and community engagement, including relevant academic support services.

“The criteria are evaluative tools that enable the institution, the audit panel and the HEQC to focus on important institutional signals and indicators of quality management.”

The first round of institutional audits, to be completed this year, comprise largely developmental reports referred to as “soft evaluations” because, even though the audit reports make recommendations and the CHE expects institutions to implement them, they are different from programme accreditation reviews. These are termed “hard evaluations” because institutions can (and do) lose accreditation and if so they are not allowed to offer programmes that are deemed not to meet certain conditions.

I have served as a panel member on several institutional audits and programme accreditation review panels and know that panel members are restricted to the specific terms and the assigned brief. It is not uncommon during audits and review processes for panel members to be drawn into other institutional matters that are outside the mandate of the panel, but in my experience, the HEQC and the panel members are careful to respect the institution’s own processes and autonomy.

These review and audit exercises are entirely dependent on a peer review process and their integrity and value depend on the professional, fair, just, consistent and ethical conduct of the panel members and of the HEQC/CHE. More often than not, there are other issues and processes under way in institutions, sometimes in the media, at the time of these audits and reviews, but these remain outside the work and activities of the panel.

The oral feedback given at the end of the week of interviews and documentation review is prepared in a very short time frame and is at best a set of impressions of possible findings. In the panels on which I have served, after the oral feedback was prepared and agreed to, all panel members are made to return hard copies and delete this information from their computers. Only the chairperson of the panel retains a copy to read out.

This is because the HEQC then ensures that the draft report in which findings appear are properly substantiated through triangulation of several sources and agreed to by the entire panel before they are released to the institution. The university is then given a chance to respond to this draft report, and the final report is served before the HEQC board before it is released.

What has taken place in the case of UKZN is that, since the presentation of the oral feedback, actions have unfolded that have corrupted and compromised the process. The university is within its rights and has a responsibility to demand that its name and image are not damaged by any process or actions undertaken by the HEQC. Not only is it unprecedented for the chair of an institutional audit panel to use confidential and privileged information outside the audit context and procedures, it is grossly unfair and unethical that the oral feedback was put into the public domain by the chair of the audit panel through the M&G when the university had not had a chance to correct errors of fact and agree on the veracity of the findings, as is accorded to all audited institutions.

This serious breach of ethics compromises the very foundation of the peer review process in which fellow academics gain access to institutions but cannot be trusted to behave within the mandate they are given or stay true to their signed word.

Writing in the M&G on the CHE’s decision to withdraw the report, Hall cited directly from the oral feedback that the CHE expressly forbids institutions to record or even to respond to. Through this serious confidentiality and ethical breach, Hall has made public untested audit findings that have not been interrogated or agreed to by the university, or approved by the HEQC board and, by so doing, may have compromised future audits.

The HEQC-appointed institutional auditors get access to confidential and sensitive information on the basis that the HEQC assures institutions that the auditors they select and appoint to act on their behalf can be trusted to respect the agreements they make and respect the institutions they are privileged to enter.

The debacle precipitated by the leak of the letter from the chair of the audit panel has taken the CHE almost two years to deal with. From the panel’s visit in 2008 to the vice-chancellor’s report on the matter to the UKZN senate and the CHE’s statement in December last year regarding the audit, the university has continued to improve its systems, policies and processes.

In the view of some experienced auditors and reviewers, the audit report has long past its sell-by date and usefulness. Senate members have also stated this. UKZN in 2011 is a very different institution from the one in 2007-2008 when the CHE institutional audit took place. With the merger firmly embedded, many new members in the leadership at school, faculty, college and executive levels and a further review and reorganisation under way to improve efficiencies, the university has continued to transform and grow.

However, the repercussions from this flouting of ethics and confidentiality on the part of Hall in releasing information in breach of the institutional audit policy and procedures will continue to reverberate in the next round of institutional audits. It raises questions about the legitimacy of the entire peer-review process, which is the bedrock of the institutional audits, particularly if the CHE drags its heels and does not act on breaches. It may well result in institutions refusing to participate in its processes because they fear that any panel member could take it upon himself to make unsubstantiated audit findings public, disregarding the signed contract and all the checks and balances of fair play and rigour that are in place.

UKZN would be within its rights to take action against the CHE and the panel chairperson for allowing this matter to come into the public domain in a way that is damaging to its image and reputation; for the CHE’s not enforcing its own regulations and procedures against panel members who breach agreements; and for its not protecting the integrity of its reports.

The implication of this kind of behaviour going into the next round of institutional audits is quite grave and it would be a pity if a process that has been collegial and self-regulated becomes bureaucratic and legalistic.

Professor Renuka Vithal is deputy vice-chancellor (teaching and learning) of the University of KwaZulu-Natal and an institutional auditor for the Council on Higher Education