Although Statistics South Africa conducts exhaustive surveys on a regular basis, an odd reliance on small, private surveys persists in the media and among politicians.
Given the economic, social and political divisions in South Africa, smaller surveys are more likely to be inaccurate, especially since many companies copy techniques from more homogeneous countries in the North.
In the absence of quality controls, findings vary widely. Commentators and policymakers may choose what to believe based on pre-existing prejudice rather than a genuine effort to find the facts.
A recent Mail & Guardian article (“Cosatu out of step”, August 19) demonstrated this problem.
It quoted Afrobarometer saying that only 5% of South Africans are active union members. This means, it argued, that the Congress of South African Trade Unions (Cosatu) cannot represent the views of many people.
But Afrobarometer’s data diverges significantly from that of the government’s Labour Force Survey, which found almost three million union members in March 2005. That comes to 10% of adults, and more than a third of formal workers.
More than 75% of miners and public servants are union members, 40% of manufacturing workers, and 20% of workers in retail and finance. In contrast, the unions are almost non-existent in farm, retail and domestic work.
So, which figures are more reliable? The answer is easy if we compare sample size. Afrobarometer’s survey covers 2 400 households; the government’s Labour Force Survey reaches more than 30 000.
Cosatu’s membership alone comes to 6% of all adults. That in itself suggests something is wrong with the Afrobarometer figures.
The same article quoted a survey of 300 people by landline telephone in the urban areas. This is a very small sample. For every metro area, it averages just 60 people. Stratifying such a minor survey effectively by income, race and gender becomes almost impossible.
In itself, the reliance on telephone surveys is dubious in South Africa. Distribution of landlines varies widely by race, income and area. According to the September 2003 Labour Force Survey, one in five Africans has a landline, compared to more than a quarter of other groups. Only one in 20 households in the former homeland areas has a landline. Any survey based on landlines is thus necessarily biased toward well-off urban minorities.
Other small, private surveys, while much loved for selective insights in some circles, harbour similar shortcomings. The Bureau of Economic Research Consumer Confidence Index is based on 2 500 households. Unisa’s Bureau of Market Research doesn’t publish its sample size in press releases, but says it relies on “past and present Unisa students, loyal to their alma mater” to cover villages. Its data has been used by the government to point to growth in the black middle class, in the teeth of much larger Statistics SA surveys that show worsening income inequalities over the past 10 years.
Certainly, small surveys have an important place in understanding social developments. But there needs to be more sophisticated quality control, including a consistent check against larger research, if they are to add to the discourse rather than distracting from it.
Neva Makgetla is coordinator of fiscal, monetary and public sector policy at Cosatu