Lofty education ideals narrow to a system driven by test results

With the introduction of the annual national assessments (ANAs) last year, which are due to be written again in grades one to six and grade nine next week, the country has joined the 81% of economically developed countries and 51% of economically developing countries that use national assessments of one kind or another.

Financial crises, rising unemployment and increasing inequalities that generate social unrest characterise the global economy, so governments worldwide are once again turning to education as a solution to these various ills and as a way to increase comparative advantages.

This is not the first time that education has been both blamed for social and economic failures and simultaneously seen as the solution, but what is new is the increasingly uniform set of ideas about the type of education required to do this and the mechanisms necessary to achieve reform.

Pasi Sahlberg of Finland calls these ideas "Germs", the backbone of the Global Educational Reform Movement. Three key premises have driven these ideas since the 1980s. These are, first, the idea that we need pupils to have creative and problem-solving skills and at the same time an increased proficiency in literacy and numeracy. The second driver is a public demand for "quality education" for all, which has led to the idea of common learning standards guided through "scripts" such as workbooks and curriculum pace-setters. Third, there is a demand for greater accountability in the education sector, which has led to the introduction of achievement targets, benchmarking, indicators and the testing of teachers.

We need to ask, though, how South Africa has moved from the lofty ideals of producing creative, independent pupils who would take the country forward into the new knowledge economies to a system that is driven by test results, a narrowed curriculum and a public disregard for the teaching profession. In major part, these responses derive from pupils' consistently poor performance in a variety of international and local achievement tests, which has been well documented and reviewed.

South Africa's dual system
Last year's national assessment results showed that "the most common percentage score pupils achieved was below 20%", the basic education department's report on the tests said – a true poverty of performance. At the grade three levels only 31% of pupils had a score of more than 50% in literacy and only 17% a score of more than 50% in numeracy.

Unsurprisingly, there is a knock-on effect to grade six. Here, only 15% had scores of more than 50% for language and 12% in mathematics. At this level, about 70% of pupils achieved less than 35%. The education minister noted in a press release announcing the results: "This is unacceptable for a nation whose democratic promise included that of education and skills development, particularly in a world that celebrates knowledge and places a premium on the ability to work skilfully with words, images and numbers."

Scores of this nature had previously been evidenced in several international assessment studies, so these outcomes were not surprising. Overall, apart from highlighting both the absolute and the relative levels of poor performance among South African pupils, these tests have, usefully, made abundantly clear the effect of poverty on performance. They have also highlighted regional differences, related in part to the relative wealth of the provinces. Significantly, the evidence indicates starkly the existence of a dual system in South Africa whereby only the pupils in the top 20% are able to achieve as expected.

This consistently poor performance has serious equity and political consequences, because failure lies most heavily on the disadvantaged; it could retard economic growth because the skills pool is weakened and hampers post-secondary school education expansion as fewer candidates become admissible to higher education.

The policy responses to this situation are crucial, given the apparent difficulties in inducing shifts in performance. The basic education department's 2010 "Action Plan to 2014: Towards the Realisation of Schooling 2025" is significant in this regard, providing targets for improvement measured against the benchmarking results of the national assessments for 2011.

However, we wish to argue that despite the assurance in the department's assessment report that "the government does not support the use of assessments for the purposes of 'naming and shaming' those who do not perform well", these prescriptions have the potential to create a true poverty of education. This is so in several senses:

Standardised teaching materials underpin assessment-driven curriculums. This leads to narrow conceptions of the goals of education and a confusion of performance with quality;

In seeking a teacher-proof curriculum, education conceived in this way underplays the pedagogical moment that produces real learning;

A focus on literacy and numeracy narrows the significance and perceived worth of other learning areas; and

Test-based accountability creates winners and losers and is likely to have detrimental consequences for equity, because schools will seek to weed out weaker pupils to achieve targets.

Consequences of this kind have been well documented in other educational systems and evidence of some of them is beginning to emerge in South Africa. For example, because the consequences for individual teachers of non-performance are unclear, the directions are emerging in proposed principal performance agreements. The converse policy is the one that financially rewards schools for achieving targets. This has the potential to encourage schools to keep weaker students from sitting public examinations. Equally insidious are the plans to close poorly performing schools.

It seems that once benchmark testing moves beyond its warrant and there is an elision between performance indicators designed for particular purposes (for example, system monitoring) and concepts of quality, these often unintended scenarios ensue and create a "hollowing out" of education. It is important then to explore the nature of indicators in education and what possibilities and limitations they afford.

Testing a system , not a pupil
First, it is important to recognise what the tests are testing to interpret the significance of indicators. Although individual students from specific schools write the tests, the results are not simply reflections of individual performances. Rather, the patterns they reveal reflect the performance of the education system as a whole. Test results are by now almost entirely predictable and performance is incontrovertibly correlated with indicators of socioeconomic status.

In this sense, comparative test scores are indicators of what research in the sociology of education has demonstrated in different countries over the past 40 years and more: that the outcomes of schooling are heavily dependent on home backgrounds or socioeconomic status.

Evidence from these studies shows that apartheid inequalities are entrenched for all but a small group of black students in desegregated urban schools. Other research shows that pupils from low socioeconomic status backgrounds are not as successful in high-achieving schools as high socioeconomic status students are. Extending this point further, there can be no doubt that South African children in poverty face the double disadvantages of poor home backgrounds and poor schools.

Because individual students and, by implication, their teachers and schools must bear the personal experience of failure, it is important to recognise that the predictable patterns of performance and failure on this scale are indications of systemic, rather than individual, failure. In other words, indicators need to be read as pointing valuably to the patterns of systemic failure and the interventions that may be needed the "can opener" function.

Schooling alone cannot remedy failure on this scale, related as it is to the inequality and socioeconomic hardship that are endemic in the broader society. This is recognised in the Organisation for Economic Co-operation and Development's report this year, "Equity and Quality in Education: Supporting Disadvantaged Students and Schools", which suggests that intervention and support are necessary and  that, "in many cases, education policies alone are not enough".

In the South African context, Michele Smith's multilevel analysis of test scores makes a similar point: "Social policies targeted at improving wellbeing in deprived neighbourhoods would have a substantial effect on educational outcome." This is critical. Indicators may play an important role in identifying where the system has problems, but they have limited value in remedying problems. Where indicator data are used punitively, or holding schools and teachers to account without providing the necessary support, they may have detrimental effects.

The second key issue is the role indicators may play in ensuring accountability. Where indicators of performance, such as tests, are in fact indicators of disadvantage, then holding schools to account for factors beyond their control will not produce the intended outcome.

Richard Elmore, who draws attention to the importance of internal capacity in school functioning, articulates this well: "Accountability systems and incentive structures, no matter how well designed, are only as effective as the capacity of the organisation to respond. The purpose of an accountability system is to focus the resources and capacities of an organisation towards a particular end. Accountability systems cannot mobilise resources that schools do not have … The capacity to improve precedes and shapes schools' responses to the external demands of accountability systems."

A case in point is teacher capacity to improve their classroom instruction. The department's report on last year's assessments noted that teacher tests confirmed that many teachers themselves did not have foundational subject knowledge and concept mastery. That being the case, it is unrealistic to expect improvement in student performance unless teachers' subject knowledge and understanding are improved.

This suggests that accountability ought to run both ways in a reciprocal process through which schools can be held to account for what it is they can do and departments can be held accountable for what it is that they should do. The recent court cases over the supply of workbooks in Limpopo suggest that this practice is not as entrenched as it might be. Indicators may then play a role on both sides.

Third, we argue that debates on indicators, together with the education systems they are part of, need to be considered in relation to broader social patterns, because it is here that possibilities for change are mainly located.

In South Africa, improvements in schooling need to be achieved in the interplay of social and political forces. The current situation is a product of apartheid and the struggles against it, but also of the policy decisions and actions by post-apartheid governments that act in the context of global neoliberalism.

The government's role
None of these can be willed away and we must work with these conditions to achieve the desired changes. In this regard, we want to suggest that the analysis and proposals of the national planning commission with regard to schooling, lining up as they do with the ideas put forward in the department's "Action Plan to 2014", are unlikely to achieve the aims they hope for.

Briefly, the planning commission diagnoses the problems in the low-quality education system as stemming from two sources: weak capacity in the civil service (from teachers and principals through to system officials) and an accompanying culture of patronage and nepotism. It is perhaps not surprising then that the planning commission proposes  that "to build technical capacity in education requires a political consensus" in which all parties stand to gain as long as they make concessions.

What is glossed over is that this would take enormous political will – in a situation in which all indicators show that this will is lacking. What is equally glossed over is the extent and depth of disadvantage in the schooling system, which, as we have suggested, is unlikely to be able to provide a route out of poverty without measures outside of the school. Put differently, despite the well-established evidence relating poverty to performance, the policy focus falls on the teachers.

We suggest, therefore, that interventions to improve school quality will need to work with teachers, unions and school communities towards shared solutions. As argued earlier, capacity building is an essential first step towards building reciprocal accountability. All indications are that changing schools takes time – and in the case of South Africa, it will require targeted intervention that supports schools and students in circumstances of poverty and hardship. This means not only appropriately targeted resources, but also resources that are appropriate – and a test-driven curriculum does not appear to be the solution here.

Education at its best
Finally, in exploring indicators and the aims of education, we argue that issues of education and poverty should be understood in ethical and political terms. In these terms, it is important to use indicators in ways that do not hollow out the aims and purposes of education. Our concern here is to work with indicators against the poverty of education and towards a more just society.

At its best, schooling is about creating spaces of teaching and learning that enable pupils to make sense of the world they share with others in ways that will enable them to change it for the better. An ethics of engagement in education entails building dispositions of enquiry, cultivating awareness and concern for a common good and facing the suffering of others with a willingness to care.

At its best, education is about enabling people to make sense of the world they share with others so that they are able to imagine alternatives and change the world for the better. These are the deeper goals of education – ones that should not be surrendered when indicators are developed.

None of the countries participating in international benchmarking tests would hold out the skills and competences these tests measure as encapsulating their full aims for education. In all cases, the aims of national education systems go far beyond such minimalist goals. Similarly, the national assessments measure basic skills and competences; they do not measure the goals of the Constitution and the ideals for education set out in various policy documents.

Tests have their value, but they also have their limits. There is always a danger – widely recognised in countries that participate in tests – that basic skills may come to dominate curriculums. Indeed, there are accounts of the perverse effects of testing in narrowing curriculums as teachers strive to achieve good results through "teaching to the test".

The accompanying danger is that habits of the mind and social values that are not expressed in the form of tests may be underplayed in ­practices that emphasise performance. Where this happens, the substance of education may be hollowed out into a performative shell that emphasises individual achievement at the expense of building a common good.

Dave Gilmour is a senior lecturer and Pam Christie a professor in the school of education at the University of Cape Town, and Crane Soudien is the university's deputy vice-­chancellor.

This is an edited version of their paper at last week's Carnegie III conference, Strategies to Overcome Poverty and Inequality, held at UCT