/ 10 January 2011

Pride — but no prejudice

Pride But No Prejudice

The National Senior Certificate (NSC) examinations are high-stakes assessments in our country. There is no exam in South Africa that generates as much public interest, analysis and review as that of the country’s grade 12 learners’ final exams. The 2010 NSC examination results, released on Thursday, are not going to be any different.

The NSC examination was introduced in 2008, when it replaced the former senior certificate. Last year was therefore the third consecutive year in which grade 12s were assessed by means of this exam. As the council for quality assurance, Umalusi believes the qualification is fast maturing and stabilising. We can be proud as a country, and as the quality council, of the manner in which this qualification has been implemented.

Internationally, it is agreed it takes up to five years for any such new qualification to stabilise. The introduction of a new qualification is also normally accompanied by a significant drop in the pass rate. In the past two years we have not seen any significant variation in the pass rate, which bears testimony to the efforts of the department of basic education and Umalusi as the quality assurer to ensure the maintenance of standards in education.

The value and credibility of a new qualification will always be viewed with suspicion. To put to rest some of these suspicions, Umalusi commissioned two major research projects. First, we started the “maintaining standards” project in 2008, which sought to benchmark the NSC in relation to the former senior certificate. This research has expanded and provides invaluable input into the standardisation processes and decisions. For 2010 this project included the evaluation of 16 subjects by teams of experts who commented on the standard of the question papers.

Second, we did significant research in collaboration with Higher Education South Africa (Hesa), the university vice-chancellors’ representative body. This research looked at how the NSC compares with international qualifications — the International Baccalaureate (IB), the Cambridge International Examinations (CIE) and the Namibian International General Certificate of Education (IGCSE).

It is pleasing to report that the NSC ranks well with both the IB and the CIE. In addition, the Umalusi-Hesa report concurs with the findings of a United Kingdom report on the status of the NSC, released in 2010, that was commissioned by the Independent Examinations Board.

Umalusi has a legal obligation to quality-assure qualifications of world-class standards: we are the custodian of the standards of the qualifications registered on our sub-framework (within the National Qualifications Framework). It is through our quality assurance of assessment and our standardisation processes that we are able to set standards for South Africa’s educational system. Umalusi continues to labour rigorously to ensure that the standard of the fairly new NSC qualification is maintained and not compromised in any way.

In quality-assuring the 2010 NSC, Umalusi implemented the following processes: Moderation of question papers; moderation of site-based assessments (that is, the assessments by schools themselves); monitoring of the conduct of examinations; verification of marking of scripts; post-exam analysis of the question papers; and standardisation of marks according to agreed statistical and ­educational principles.

To carry out these quality assurance processes Umalusi used the services of teams of subject specialists, referred to as external moderators and evaluators, as well as monitors who are conversant with the examination system. The standardisation is undertaken by the assessment standards committee — which is a committee of the Umalusi Council. The members of this committee are esteemed statisticians and educationists.

Two key initiatives for 2010 were put in place by Umalusi’s quality of assurance of assessment unit:

  • Approval of question papers
    Umalusi introduced an additional layer of quality assurance of question papers. This process entailed scrutiny of all the approved question papers to ensure that the cognitive demand and difficulty level of the question papers are in line with the policy prescripts. This process proved to be very useful in strengthening the already rigorous quality assurance regime.
  • Site-based assessments
    Umalusi piloted a revised approach to the moderation of site-based assessments. The revision followed the abolition of learners’ portfolios as evidence of performance. The outstanding feature of this approach is that instead of moderating the entire learner portfolio, as was the case previously, the 2010 moderation focused on ‘specified” tasks. Umalusi requested provinces to submit specified tasks per subject to ascertain whether the correct assessment methods and tools were used to assess the different tasks.

The 2010 process then culminated in the standardisation of results. The process is often misunderstood. Standardisation involves processes used the world over to mitigate the effect on a learner’s performance of factors other than the learner’s knowledge and aptitude. It is important to note also that standardisation was employed by both of Umalusi’s predecessors (the Joint Matriculation Board from 1918 to 1992 and the South African Certification Council from 1992 to 2001).

A large-scale examination system such as the NSC inevitably has many sources of variability. These can include levels of difficulty across question papers, errors in papers and inconsistency in the marking. Standardisation works on the assumption that, for large populations, the distribution of aptitude and intelligence does not change appreciably from one year to the next and that it is therefore reasonable to expect that, all things being equal, this year’s cohort of learners should perform at a level comparable with last year’s cohort.

Standardisation seeks to mitigate fluctuations in learner performance that result from factors within the examination processes themselves rather than from the knowledge, aptitude and abilities of the learners. Umalusi considers that this process was characterised in 2010 by a high degree of professionalism and that the adjustment discussions were premised on sound educational principles.
Based on the various qualitative reports presented to us, we concluded that the 2010 examinations were credible. There is currently no report of any irregularities that would compromise the credibility of the 2010 NSC.

Dr Mafu Rakometsi is chief executive of the Umalusi Council

Measures of quality

Last week the quality watchdog body Umalusi announced it had approved the release of the National Senior Certificate results for the 641 533 candidates, in 58 subjects, who wrote the exam.

Key factors were that:

  • Raw examination marks were accepted for 39 subjects;
  • Upward adjustments were made in nine subjects; and
  • Downward adjustments were made in 10 subjects.

Umalusi said it was “pleased with the fact that for 67,24% of the subjects raw examination marks were accepted. This is an indication that the qualification is stabilising and that the assessment instruments are, in general, being pitched at the correct levels.”

The quality body said that in the subjects where adjustments had been made, “no marks were adjusted to the maximum of 10% provided for in the standardisation principles”.

Umalusi also explained the 5% mark allocation for “language compensation”, observing that this is “the most misunderstood concept in this country”:
“It is not the case that some learners obtain a bonus of 5% in the exam results,” Umalusi said.

“In terms of the policy on language compensation, learners who offer an African language as home language and do not offer Afrikaans or English as home language qualify for a 5% language compensation on the mark they have obtained in a non-language subject.

“For example a learner who obtains a mark of zero out of 300 will obtain 5% of zero (which is zero) for language compensation; a learner who obtains 10 out of 300 will receive 5% of 10, which is 0.5 marks, for language compensation; a learner who obtains 100 out of 300 marks will obtain an additional five marks for language compensation.” — David Macfarlane