/ 3 June 2011

Flaws in small business data

For a sector cursed with probably the worst “lip-service to action” ratio in South Africa, it wasn’t a good outcome: two major research projects into the number of small businesses in South Africa have come up with such vastly different figures that it may well stun already confused economic development officials into catatonia.

The latest report from the Global Entrepreneurship Monitor (Gem) asserts that one in nine South Africans runs a business of some sort, while last year’s Finscope study, the biggest small-business survey in South Africa yet, counted one in six. This is a difference of two million businesses.

While it further clouds the waters of the poorly understood small-business sector, the discrepancy does throw light on quite a different matter: South African economic research and its use by government and its development agencies.

In a country with a workforce of 17-million and 40% unemployment, the over- or undercount of two million businesses is no marginal figure, even if they are mostly start-ups or self-employed survivalists.

And with government’s declared focus on job creation, one would think that every national government economic planner or provincial policy wonk would sit up and ask, “what’s going on here?”, or at least take sides and defend one set of research results over the other.

But no one from the department of trade and industry (which part-funded both research projects), the department of economic development (tasked with the crucial role of setting up a new small-business finance agency for South Africa), the DTI’s Small Enterprise Development Agency, the treasury, the National Planning Commission, Statistics South Africa, or any provincial government or development corporation has asked anybody in government or academia to look into the discrepancy.

According to Finscope researcher Irma Grundling, only one NGO, the Business Trust, has so far requested a comparative analysis of the two sets of data, which will be done in the next few months. The Gem researchers were not aware of anyone ­comparing the research.

Although it is true that the latest annual Gem report was released only last month, reflecting field research done just before the World Cup last year, economic development experts in South Africa have been staring the discrepancy in the face since the release of the Finscope survey last year. The contrast with the previous year’s Gem results was even starker, with Gem counting one in 14 South Africans in business — fewer than half of Finscope’s one in six.

Perhaps the most worrying aspect is the lack of engagement between the two research teams themselves. They should be acutely interested in each other’s work, but in interviews with the researchers a distinct lack of enthusiasm for getting involved in a debate with the other side came to the fore. All admitted that they were only vaguely familiar with the methodology and paradigm used by the other side.

Both sets of researchers started by claiming that the other team’s research couldn’t be compared with theirs because it was measuring something else and had an altogether different purpose.

Finscope’s Grundling claimed, for example, that the Gem research looked for entrepreneurial intent and not just any business activity. It had to be pointed out to her that the Gem project coined the terms “necessity and opportunity entrepreneur” and counted anybody involved in a business, whether for profit or survival.

On further questioning, however, the researchers admitted that the differences could lie in the different methodologies used.

Gem, for example, contracts Nielsen SA to interview a random adult population sample of about 3 000 South Africans, business owners and non-business owners, and extrapolates its numbers from there.

Finscope, on the other hand, did household surveys in areas statistically representative of the South African population and interviewed more than 6 000 people involved in business activity. It then used the interval between business owners (the number of people approached before a business owner is found among them) to extrapolate ­universal numbers for South Africa as a whole.

Both methods, if accurately used, should come up with roughly the same number of business owners in South Africa — provided the definition of a business owner they use is the same.

Finscope relies on self-identification of its respondents as business owners. Gem, in contrast, makes sure through various questions asked in slightly different ways that respondents are actually busy with some form of income-generating activity before counting them as entrepreneurs, said Gem co-researcher Jackie Kew.

No one has yet analysed the impact of this difference in definition on the numbers. Another crucial difference that cries out to be analysed is Finscope’s counting of foreigners, who, they reckon, account for 17.3% of the entrepreneurial activity in South Africa. At 967 000 individuals this is such a serious number that it should be sending policy developers, from treasury to home affairs, scrambling for more information. Gem counts only South Africans.

Furthermore, Finscope counts South Africans from the age of 16, Gem from 18. Entrepreneurs younger than 18 make up 2% of Finscope’s number.

Kew said part of the problem is that much research in South Africa “happens in silos”, with not enough resources spent on meta-analyses. Gem, for example, forges short-term contracts with its researchers, who are too busy to conduct further research into why their results differ from those of other surveys.

The question, therefore, becomes why no one in government, the ultimate user of such research, is having a proper analysis done. In an emailed response to questions, the chief executive of the Small Enterprise Development Agency (Seda), Hlonela Lupulwana, wrote: “We believe there are lessons to be learned from both reports.”

She did not seem bothered by the fact that such an enormous difference between the basic outcome of the two reports — the sheer number of purported business owners in South Africa – may be due to skewed sampling, and therefore disqualify other conclusions drawn from one or the other of the reports.

“The research reports provide [such] a wide range of information — over and above the number of small enterprises that exist in the economy — that we would not want to summarily dismiss [them] by labelling any of the reports untrustworthy,” she wrote.

Insiders at the department of economic affairs who are involved in setting up a new small-business finance agency from a merger of the Industrial Development Corporation’s small-business unit, the struggling Khula Enterprise Finance and the DTI’s micro-finance Apex fund, don’t seem to be using any of the research. Awareness levels of the research and the differences between the two reports were very low.

Both Gem and Finscope show that jobs are created by sophisticated small businesses, not informal ones. Asked if this will help the department of economic development to prioritise the scarce recourses of the new finance agency, an official, who spoke anonymously, said: “Well, job creation is a priority for us, but so is helping people at the lowest part of the economy get into the formal economy. We can’t neglect the one and favour the other, so it’s a sort of a both kind of approach.”

The statement seems to encapsulate government’s attitude to hard research into the small-business sector. It is occasionally commissioned and half acknowledged, but when it comes to setting policy, ideology trumps data, even if it is the “pay lip-service to everyone” ideology characteristic of Jacob Zuma’s administration.

Although no direct research has been done into the question, all data seem to suggest that informal businesses simply don’t graduate to become formal ones, no matter how much money government agencies spend on their training, mentoring, finance and business-plan development. What the Gem and Finscope reports show is that if, one day, someone does get around to studying the question, there is no guarantee that anyone will take it seriously.