Successes or failures, we need evidence

This year has been declared the International Year of Evaluation by the United Nations. The resolution, asking member countries to strengthen their evaluation capacity and to report on progress, was supported by 41 countries. South Africa was not a signatory, despite the participation of its Brics partners, Brazil, Russia, India and China.

It’s not clear why South Africa didn’t sign. There was no mention of the resolution or evaluation in the State of the Nation address, despite the international cry for more accountability and for reform and development to be informed by evidence-based research and for the direct involvement of recipients in policy-making. This is what the Year of Evaluation hopes to achieve.

We have a relatively new but active department of performance monitoring and evaluation in the ministry of the presidency. The department has developed a government-wide monitoring and evaluation policy framework and a national evaluation policy framework; it supports similar frameworks for the treasury and South Africa’s statistical quality assessment framework. The Public Service Commission undertakes systematic reviews of government performance at all levels. South Africa is also a willing participant in the African Peer Review Mechanism. It may be felt by some that we are doing enough in this area.

We also have an active national association, the South African Monitoring and Evaluation Association (Samea), established in 2005. It has held successful conferences with participation by international and African role players, civil society and government.

Global experts have shared knowledge and experience through training opportunities for South Africans, and there has been substantial growth in academic courses on the issue. Samea members serve in international networks of this kind, and the association has a close working relationship with some government departments. This has become a model for other countries to emulate.

The government’s approach to monitoring and evaluation has an evidence-based “philosophy”: policymakers are encouraged to use the best scientific evidence in devising policy to address specific problems. The government initially focused on specific (strategic) outcomes such as quality basic education, reduced crime and a healthy population, making them the focus of government policies and action.

But more can be done to inform the public about government initiatives, objectives, approaches, successes and challenges – particularly those related to evaluation.

Monitoring and evaluation have multiple aims: to inform audiences about the effect of an intervention; to assist government to address the accountability criteria of a specific policy or programme; to improve, clarify or develop an intervention.

Yet evaluations are sometimes only used for symbolic purposes – for show – to give the appearance of action. Scholar Carol H Weiss also noted the existence of “evaluation as subterfuge”, such as the commissioning of an evaluation study as a way to delay decision-making. Some of South Africa’s commissions of inquiry fall into this category.

A major challenge for evaluation is that it operates within a political environment, which can be complex and contested, with multiple stakeholders drawing on different histories and loyalties and informed by contradictory values. Government programmes are created by political forces; decisions and investments are made by cadres loyal to certain political ideals. Any evaluation will have political connotations. A purist, scientific view seeks a value-free approach, one uncontaminated by politics, authority and ideology.

But evaluation is a political as well as a scientific practice. It is up to the evaluator to negotiate and agree on the values the practices will adhere to, and the utility factors to which they will subject the findings to.

Evidence-based policy development has led some governments (thankfully, not South Africa) to see randomised trials as the gold standard for evidence in public policy, despite evidence to the contrary. Such policy development and assessment can improve the quality of policy decisions, but they are based on many factors. Reliance on knowledge produced by experts only can exclude the voices of communities on the ground – ironically, the chief recipients of the relevant programmes.

Service delivery protests are one sign of dissatisfaction: people want better conditions of living; they want attention to be given to specific community issues. Growing informal settlements, continued urbanisation, the growth in unemployment and the apparent failure of schools to address education needs are among the “evidence” ordinary people use to inform their views of our government’s performance. The government, the not-for-profit sector and corporations must address these issues – and produce and share evidence that could change citizens’ views.

In this, President Jacob Zuma’s address missed a major opportunity. We need to send a strong signal to other developing countries that we support more accountability.

We must create mechanisms to engage with communities in assessing interventions and programmes for their effectiveness, to improve them where necessary and possible, to develop new interventions to address communities’ actual needs, and to end programmes that fail to meet their objectives. The purposes of the evaluations must be clear, unambiguous and not for show.

Dr Mark A Abrahams is the editor-in-chief of the African Evaluation Journal

These are unprecedented times, and the role of media to tell and record the story of South Africa as it develops is more important than ever. But it comes at a cost. Advertisers are cancelling campaigns, and our live events have come to an abrupt halt. Our income has been slashed.

The Mail & Guardian is a proud news publisher with roots stretching back 35 years. We’ve survived thanks to the support of our readers, we will need you to help us get through this.

To help us ensure another 35 future years of fiercely independent journalism, please subscribe.


ConCourt settles the law on the public protector and interim...

The Constitutional Court said it welcomed robust debate but criticised the populist rhetoric in the battle between Busisiwe Mkhwebane and Minister Pravin Gordhan

Where is the deputy president?

David Mabuza is hard at work — it’s just not taking place in the public eye. The rumblings and discussion in the ANC are about factions in the ruling party, succession and ousting him

Press Releases

Covid-19 and Back to School Webinar

If our educators can take care of themselves, they can take care of the children they teach

5G technology is the future

Besides a healthcare problem Covid-19 is also a data issue and 5G technology, with its lightning speed, can help to curb its spread

JTI off to court for tobacco ban: Government not listening to industry or consumers

The tobacco ban places 109 000 jobs and 179 000 wholesalers and retailers at risk — including the livelihood of emerging farmers

Holistic Financial Planning for Professionals Webinar

Our lives are constantly in flux, so it makes sense that your financial planning must be reviewed frequently — preferably on an annual basis

Undeterred by Covid-19 pandemic, China and Africa hold hands, building a community of a shared future for mankind

It is clear that building a community with a shared future for all mankind has become a more pressing task than ever before

Wills, Estate Administration and Succession Planning Webinar

Capital Legacy has had no slowdown in lockdown regarding turnaround with clients, in storing or retrieving wills and in answering their questions

Call for Expression of Interest: Training supply and needs assessment to support the energy transition in South Africa

GIZ invites eligible and professional companies with local presence in South Africa to participate in this tender to support the energy transition

Obituary: Mohammed Tikly

His legacy will live on in the vision he shared for a brighter more socially just future, in which racism and discrimination are things of the past

The best local and international journalism

handpicked and in your inbox every weekday