Who is that face behind the mask?

Facial recognition technology is ubiquitous — it’s in smartphones and buildings and used in banking and by the police. 

But, because of Covid-19 and the need to wear masks, facial recognition has been restricted. The masks cover lower parts of the face, which the technology needs to identify a person. 

But scientist Ismael Msiza, 35, has made a breakthrough. By using conventional image processing techniques and artificial intelligence methods, he has created a prototype that is more than 90% accurate even if a person is wearing a mask. 

Msiza grew up in Kameelrivier in Mpumalanga and moved to Johannesburg in 2003 to study electrical engineering. Midway he found he was more interested in information systems and computer science and the Council for Scientific and Industrial Research (CSIR) funded his studies. His interests also include software engineering, machine learning and systems analysis in various domains, including biometric identity management. 

By 2011 he was part of the CSIR’s biometrics research group where he developed a structural fingerprint classifier that works with only partial information, and introduced novel fingerprint features. 

Fast forward to the pandemic. “It was very important for us to restore the dignity of our biometric recognition systems,” said Msiza because the biometric system was brought to its knees by Covid-19. 

Fingerprint recognition lost its popularity because people are not allowed to touch surfaces while facial recognition also lost its prevalence because a mask prevents accuracy. 

Facial recognition involves biometric identification that uses a person’s features to verify their identity. The technology collects a set of unique biometric data of each person’s face. 

Msiza said that people think the technology can simply reconstruct the hidden facial features or remove the masks in some artificial way. The technology instead focuses on the features that are left exposed by the mask, such as the forehead and the eyes.

“What makes it difficult is that you are left with less than half of the facial landmarks exposed,” he said.

Msiza is the head of a technical service house that started working on this facial recognition prototype in May. He and his team do not have the finances to develop it further, and are now trying to sell it to original equipment manufacturers (OEMs) such as Samsung, Apple and Suprema. 

“OEMs that we have approached are showing interest,” said Msiza. 

According to a preliminary study by the National Institute of Standards and Technology released in July, identifying an individual wearing a mask was done “with great difficulty”. 

The study found that algorithm accuracy with masked faces declined substantially across the board. Using unmasked images, the most accurate algorithms fail to authenticate a person, about 0.3% of the time. 

Masked images raised these top algorithms’ failure rate to about 5%, while many otherwise competent algorithms failed between 20% to 50% of the time.

To get around this obstacle of the mask, Msiza used a hybrid solution that combines conventional image processing techniques and artificial intelligence methods. 

He said that his prototype uses deep learning and machine learning in artificial intelligence. This kind of technology is capable of learning unsupervised data from unstructured or unlabelled material. 

It uses algorithms inspired by the human brain to learn from large amounts of data. Essentially, just like humans learn from experience, the deep learning algorithm performs a task repeatedly, each time refining it a little to improve the outcome.

Tshegofatso Mathe is an Adamela Trust business reporter at the M&G

Subscribe to the M&G

These are unprecedented times, and the role of media to tell and record the story of South Africa as it develops is more important than ever.

The Mail & Guardian is a proud news publisher with roots stretching back 35 years, and we’ve survived right from day one thanks to the support of readers who value fiercely independent journalism that is beholden to no-one. To help us continue for another 35 future years with the same proud values, please consider taking out a subscription.

Tshegofatso Mathe
Tshegofatso Mathe
Tshegofatso Mathe is a financial trainee journalist at the Mail & Guardian.

Related stories


Subscribers only

Poachers in prisons tell their stories

Interviews with offenders provide insight into the structure of illegal wildlife trade networks

Covid-overflow hospital in ruins as SIU investigates

A high-level probe has begun into hundreds of millions of rand spent by the Gauteng health department to refurbish a hospital that is now seven months behind schedule – and lying empty

More top stories

The politics of the Zuma-Zondo showdown

Any move made by the Zondo commission head or by former president Jacob Zuma must be calculated, because one mistake from either side could lead to a political fallout

Museveni declared winner of disputed Uganda election

Security personnel out in force as longtime president wins sixth term and main challenger Bobi Wine alleges rigging.

Pay-TV inquiry probes the Multichoice monopoly

Africa’s largest subscription television operator says it is under threat amid the emerging popularity of global platforms like Netflix and Amazon Prime

​No apology or comfort as another Marikana mother dies without...

Nomawethu Ma’Bhengu Sompeta, whose funeral will be held this weekend, was unequivocal in calling out the government for its response to the Marikana massacre

press releases

Loading latest Press Releases…