This was the sobering view from analysts and commentators after the 2012 results were released by Basic Education Minister Angie Motshekga on Wednesday.
"We shouldn't be fixated on numbers alone. We need to investigate how well our matrics are doing instead of how many are passing," Professor Ruksana Osman, head of Wits University's School of Education said on Wednesday.
At an aggregate of 73.9%, 2012's pass rate indicates a year-on-year increase of 10.3 percentage points in the number of learners who passed matric since 2008.
National grade 12 mathematics and physical science results also saw an encouraging increase in comparison to 2011.
Mathematics pass rates nationally were 54%, up from 46.3% and physical science rose to 61.3% from 53.4%.
Last year's matric results also saw a 2.3 percentage-point increase in learners obtaining a university exemption, from 24.3% to 26.6%.
But Osman said while improved pass rates are encouraging, it is not strictly a sign of an improving education system.
"It's pleasing to see an increase in the overall pass rate, but what type of quality is being produced?" she added.
Osman's questions were echoed by Equal Education – South Africa's biggest education related non-governmental organisation.
"Many questions need to be asked that the current data doesn't answer," Doron Isaacs, deputy secretary general of Equal Education said.
"We need to establish if our students are just scraping through, particularly in problem areas, so that steps can be taken to rectify the situation."
Isaacs suggested the results may be indicative of a drive to achieve increased pass rates without adequately addressing quality aspects in basic education.
"In every school countrywide there has been pressure to see an improved matric pass rate," he said.
"There is no telling how this affects the level of education being offered and if an attempt was made to get the best out of all learners."
Basil Manuel, president of the National Professional Teachers' Organisation, South Africa's second-largest education union, also said there needed to be a focus on "quality over quantity".
"The focus on pass rates can be very misleading," Manuel said. "The real focus should be on the quality of passes we are achieving and what we can do to improve them."
Annette Lovemore, Democratic Alliance spokesperson on basic education, said a broader approach towards the assessment of matric pass rates was needed.
"We have to cease from using inaccurate indicators," she said.
"It is encouraging to see an increase in the pass rate but we need to look at the number of failures and where they are coming from in order to address the massive number of dysfunctional schools our learners find themselves in," Lovemore added.
She said this would result in the "systematic improvement of our education system".
After the presentation of the results, basic education director general Bobby Soobrayan acknowledged the need for better quality passes at matric level.
"The standard we are achieving will only get better if progress is made in improving the capacity of all of our schools in the system – especially those in poorer areas," Soobrayan said.
Motshekga announced in the Government Gazette this past November that she would form a task team to investigate the standard of South Africa's National Senior Certificate exams in 2013.
The task team would compare the country's matric standard and current pass benchmarks with countries on the same level of development, the Gazette said..