Get more Mail & Guardian
Subscribe or Login

Successes or failures, we need evidence

This year has been declared the International Year of Evaluation by the United Nations. The resolution, asking member countries to strengthen their evaluation capacity and to report on progress, was supported by 41 countries. South Africa was not a signatory, despite the participation of its Brics partners, Brazil, Russia, India and China.

It’s not clear why South Africa didn’t sign. There was no mention of the resolution or evaluation in the State of the Nation address, despite the international cry for more accountability and for reform and development to be informed by evidence-based research and for the direct involvement of recipients in policy-making. This is what the Year of Evaluation hopes to achieve.

We have a relatively new but active department of performance monitoring and evaluation in the ministry of the presidency. The department has developed a government-wide monitoring and evaluation policy framework and a national evaluation policy framework; it supports similar frameworks for the treasury and South Africa’s statistical quality assessment framework. The Public Service Commission undertakes systematic reviews of government performance at all levels. South Africa is also a willing participant in the African Peer Review Mechanism. It may be felt by some that we are doing enough in this area.

We also have an active national association, the South African Monitoring and Evaluation Association (Samea), established in 2005. It has held successful conferences with participation by international and African role players, civil society and government.

Global experts have shared knowledge and experience through training opportunities for South Africans, and there has been substantial growth in academic courses on the issue. Samea members serve in international networks of this kind, and the association has a close working relationship with some government departments. This has become a model for other countries to emulate.

The government’s approach to monitoring and evaluation has an evidence-based “philosophy”: policymakers are encouraged to use the best scientific evidence in devising policy to address specific problems. The government initially focused on specific (strategic) outcomes such as quality basic education, reduced crime and a healthy population, making them the focus of government policies and action.

But more can be done to inform the public about government initiatives, objectives, approaches, successes and challenges – particularly those related to evaluation.

Monitoring and evaluation have multiple aims: to inform audiences about the effect of an intervention; to assist government to address the accountability criteria of a specific policy or programme; to improve, clarify or develop an intervention.

Yet evaluations are sometimes only used for symbolic purposes – for show – to give the appearance of action. Scholar Carol H Weiss also noted the existence of “evaluation as subterfuge”, such as the commissioning of an evaluation study as a way to delay decision-making. Some of South Africa’s commissions of inquiry fall into this category.

A major challenge for evaluation is that it operates within a political environment, which can be complex and contested, with multiple stakeholders drawing on different histories and loyalties and informed by contradictory values. Government programmes are created by political forces; decisions and investments are made by cadres loyal to certain political ideals. Any evaluation will have political connotations. A purist, scientific view seeks a value-free approach, one uncontaminated by politics, authority and ideology.

But evaluation is a political as well as a scientific practice. It is up to the evaluator to negotiate and agree on the values the practices will adhere to, and the utility factors to which they will subject the findings to.

Evidence-based policy development has led some governments (thankfully, not South Africa) to see randomised trials as the gold standard for evidence in public policy, despite evidence to the contrary. Such policy development and assessment can improve the quality of policy decisions, but they are based on many factors. Reliance on knowledge produced by experts only can exclude the voices of communities on the ground – ironically, the chief recipients of the relevant programmes.

Service delivery protests are one sign of dissatisfaction: people want better conditions of living; they want attention to be given to specific community issues. Growing informal settlements, continued urbanisation, the growth in unemployment and the apparent failure of schools to address education needs are among the “evidence” ordinary people use to inform their views of our government’s performance. The government, the not-for-profit sector and corporations must address these issues – and produce and share evidence that could change citizens’ views.

In this, President Jacob Zuma’s address missed a major opportunity. We need to send a strong signal to other developing countries that we support more accountability.

We must create mechanisms to engage with communities in assessing interventions and programmes for their effectiveness, to improve them where necessary and possible, to develop new interventions to address communities’ actual needs, and to end programmes that fail to meet their objectives. The purposes of the evaluations must be clear, unambiguous and not for show.

Dr Mark A Abrahams is the editor-in-chief of the African Evaluation Journal

Subscribe for R500/year

Thanks for enjoying the Mail & Guardian, we’re proud of our 36 year history, throughout which we have delivered to readers the most important, unbiased stories in South Africa. Good journalism costs, though, and right from our very first edition we’ve relied on reader subscriptions to protect our independence.

Digital subscribers get access to all of our award-winning journalism, including premium features, as well as exclusive events, newsletters, webinars and the cryptic crossword. Click here to find out how to join them and get a 57% discount in your first year.

Related stories


If you’re reading this, you clearly have great taste

If you haven’t already, you can subscribe to the Mail & Guardian for less than the cost of a cup of coffee a week, and get more great reads.

Already a subscriber? Sign in here


Subscribers only

‘The children cannot cope any more’: Suicide in Calvinia highlights...

How Covid-19 has intensified the physical and emotional burdens placed on children’s shoulders.

Capitec Bank flies high above Viceroy’s arrow

The bank took a knock after being labelled a loan shark by the short seller, but this has not stymied its growth

More top stories

Farmers squeezed by big retailers

It may be beneficial for consumers when supermarkets push to get the lowest price from suppliers, but it can harm the farmers

Sisters pave the way with ecobricks

The durable bricks are made from 30% recycled plastic, some of which they collect from a network of 50 waste pickers

If the inflation-driving supply strain in the US lasts, it...

In South Africa, a strong trade surplus, buoyed by robust commodity prices, will cushion our economy against pressure arising from US policy

Covid-19: No vaccine booster shots needed yet

Scientists agree it is important to get most of the population vaccinated before giving booster jabs

press releases

Loading latest Press Releases…