/ 19 April 2013

SA’s maths and science: When are stats not stats?

African librarians need data skills to make the transition to open access.
African librarians need data skills to make the transition to open access.

When are statistics not statistics? When no one bothers to check how they were arrived at.

This is the case of the statistics extracted by the Democratic Alliance – and parroted by news organisations – from the World Economic Forum report that placed South Africa 140th out of 144 countries for the quality of our education system and 143rd out of 144 for the quality of our maths and science education.

While the problems of the South African education system are not a state secret, the use of this report to highlight the problem is a failure of the first level of critical thinking, both on the part of the DA and of the news organisations that have repeated the information.

Let's break it down so we can understand why there is a problem with these statistics.

The base information was published in the WEF's "Global Competitiveness Report 2012-2013" in 2012. So the information that is being touted as news is hardly new. This information was gathered by the WEF through a survey of executives in the countries concerned.

In the report, it recognises its partners in the countries surveyed and in South Africa they were Business Unity South Africa and Business Leadership South Africa.

The report reveals that the average number of executives surveyed in each country was 100, but in SA only 45 contributed to the survey. By contrast 89 executives in Lesotho contributed to the survey.

The questions asked were: "How would you assess the quality of maths and science education in your country's schools?" and "How well does the educational system in your country meet the needs of a competitive economy?"

The answers show that it is not the WEF ranking South Africa's education system, but rather the WEF giving a ranking based on the perceptions of executives in the countries concerned.

While it does provide some insight into the state of South Africa's education system, it is vulnerable to personal bias and the perception prevalent in the country that the education system is fundamentally broken. This is a perception that the government has done little to lessen because of its lackluster handling of the textbook crisis last year and its inability to provide basic facilities to many of our poorer schools.

The questions asked are also vulnerable to the effect of perception in other countries. If the people in Lesotho, which has a very different economy to South Africa, perceive that its education system is meeting the needs of business then they will rank it highly. Countries with education systems that are on the rise are going to rank it higher than countries where there has been a lot of negative news about the education system.

The fact that executives in Brazil pushed the maths and science score in that country down to 132nd place indicates that we probably shouldn't be quite so hard on ourselves. Not to mention the 40 to 50 countries that weren't even part of the survey, many of them far less developed than South Africa.

The report that the DA statement references "The Global Information Technology Report 2013" actually has a lot of good news for South Africa. In some places we rank as high as 17th for the "Efficiency of legal system in settling disputes", 15th for the "Quality of management schools" and 36th for "Business-to-business Internet use", with an overall ranking of 70th, second only to Mauritius in Africa.