/ 7 December 2018

Education analysis misses the mark

The improvements we are witnessing are being driven by the historically disadvantaged parts of the schooling system
The improvements we are witnessing are being driven by the historically disadvantaged parts of the schooling system, says the writer (Graphic: John McCann)

A recent comment piece published in the Mail & Guardian (“Basic education is failing the economy”) expresses alarm at the performance of the schooling system and the implications for the economy. We share the fundamental concern that the school system still needs much improvement in order to contribute to economic growth and transformation, but unfortunately the analysis is weak.

First, the glaring factual errors indicate perfunctory knowledge of key data sources and education in the country. For example, the authors assert that the “latest results” available from international assessments of education quality come from the national school effectiveness study (NSES), but these are by no means the latest results nor was it an international study.

The NSES was undertaken almost 10 years ago, between 2007 and 2009, in a sample of eight provinces in South Africa. How then do the authors cite average numeracy scores for other countries? They then introduce the Trends in International Mathematics and Science Study (Timss) but go on to quote statistics that come from a different study. Tanzania, for example, was not even part of the study.

The article also draws on the Southern and Eastern African Consortium for Monitoring Education Quality (SEACMEQ) to illustrate problems of teacher subject knowledge. The authors argue that “60% of maths teachers, teaching from grades one to six, failed to pass tests for maths at that grade level”. This reflects a clear gap in knowledge because they have not fully understood how these assessments are administered.

These tests were only administered to a sample of grade six teachers. These teacher tests were also not at the “grade level” — they were generally more difficult than the tests written by grade six pupils, and included only a few overlapping items.

The authors of the article had referred to the flawed and widely discredited World Economic Forum 2016-17 education ranking, which is based on an opinion survey among a small sample of business executives in the various countries, when more systematic and scientific methods are available for measuring cross-country performance.

The misunderstanding of the data sources referred to in the article reveals a lack of working experience in this sector, and that only a careless reading of other reports could have informed this article. There is also no proper referencing of the reports, which were misquoted.

The second point of concern is that there is a lack of nuance in understanding South Africa’s journey since 1994. The expansion of access to education and pro-poor policies are not insignificant achievements.

A major focus of policy and planning since 1994 has been on introducing and extending pro-poor programmes. These include the introduction of no-fee schools since 2007 so that about two-thirds of children now do not pay fees, daily meals to more than nine million children through the national school nutrition programme are provided, and the expansion of the learner transport programme allows children in remote areas to arrive at school safely, on time and ready to learn.

Educational participation among five-year-olds in South Africa has increased from about 40% in 2002 to nearly 90% in 2017 (according to Statistics SA General Household Surveys).

Primary school and secondary school completion rates have increased considerably over time, especially among black Africans. The number of matric passes has increased from 191 000 in 1994 to more than 401 000 in 2017.

The article presents an illusion that education outcomes in other countries instantaneously improve because of policy reforms. But such reforms generally take time to implement and even longer to show a meaningful change in outcomes, and it is not true that we have seen spectacular increases in learning outcomes in other countries.

South Africa has been the fastest-improving country in the Timss grade nine maths and science assessments between the surveys of 2002, 2011 and 2015, and has registered improvements in all three international assessments of education quality in which we participate, that is, Timss, the SEACMEQ and the Progress in International Reading and Literacy Study.

Of course, we acknowledge that the overall level of achievement in key learning foundations is still low, and that is a binding constraint to further improvements in matric outputs, post-schooling opportunities and labour market outcomes. We also acknowledge that there are still critical inequalities in educational outcomes along racial and socioeconomic lines. But this has to be understood in the context of improvement and a necessarily long-term development challenge, which is not unique to South Africa.

The improvements we are witnessing are being driven by the historically disadvantaged parts of the schooling system. For example, the Timss improvements since 2002 were largest at the bottom end of the performance distribution and in poorer provinces.

The annual numbers of black and coloured national senior certificate (NSC) passes for bachelors degrees have roughly doubled since 2008, and the numbers of white and Indian NSC for bachelors degrees were stagnant over the same period, which indicates that the former improvement is genuine and unlikely to be reflective of changing standards.

The grade 12 maths and physical science results also reveal an increase in the number of pupils reaching critical performance thresholds in these gateway subjects, and such increases are particularly seen in the number of black pupils who obtain the critical threshold in maths — reflecting that this trend has been towards diminishing racial inequalities.

Last, the recommendations put forth by the authors for how to improve our education system are vague and reflect a weak grasp of the sector. The authors, as researchers at the Financial and Fiscal Commission, should be aware of how complex it is to change education policy and manage the schooling system in a manner that brings about better learning outcomes.

The political context of strong unions, capacity constraints in the various organisations that contribute to research and policymaking in education and, in the past few years, serious budget constraints, present difficult obstacles.

Yet we are in a better position today than we were, say, 15 years ago.

Results have improved, as already described. The National Development Plan (NDP) provides a set of values and priorities that stakeholders can rally around. We appreciate that we need to prioritise evidence-based interventions to improve literacy and numeracy among primary school pupils.

The challenge for those making recommendations about education policy now is to provide greater detail about how to heed the NDP’s call to finance policies that are oriented towards improvements in what pupils learn.

The NDP’s and the department of basic education’s specific areas of action in this regard include clearer job descriptions for school principals, a new national integrated assessment framework, new evidence-based approaches to teaching reading in the early grades such as the early grade reading study, a “results-oriented mutual accountability” system encompassing schools, district offices and communities, an “inclusive basket of indicators” for grade 12 to ensure that schools do not simply push pass rates up by forcing weaker pupils out, and the introduction of a three-stream model, focusing on academic, technical vocational and technical occupational pathways to address the shortage of skills in a changing world.

A challenge for all who participate in the education policy debates is to take good general ideas and translate them into workable policy solutions.

Jesal Kika and Dr Stephen Taylor work for the department of basic education and are responsible for research, monitoring and evaluation