Unmasked: Ismael Msiza and his team used a hybrid solution that combines conventional image processing and artificial intelligence to enable facial recognition of people wearing Covid masks. (James Puttick)
Facial recognition technology is ubiquitous — it’s in smartphones and buildings and used in banking and by the police.
But, because of Covid-19 and the need to wear masks, facial recognition has been restricted. The masks cover lower parts of the face, which the technology needs to identify a person.
But scientist Ismael Msiza, 35, has made a breakthrough. By using conventional image processing techniques and artificial intelligence methods, he has created a prototype that is more than 90% accurate even if a person is wearing a mask.
Msiza grew up in Kameelrivier in Mpumalanga and moved to Johannesburg in 2003 to study electrical engineering. Midway he found he was more interested in information systems and computer science and the Council for Scientific and Industrial Research (CSIR) funded his studies. His interests also include software engineering, machine learning and systems analysis in various domains, including biometric identity management.
By 2011 he was part of the CSIR’s biometrics research group where he developed a structural fingerprint classifier that works with only partial information, and introduced novel fingerprint features.
Fast forward to the pandemic. “It was very important for us to restore the dignity of our biometric recognition systems,” said Msiza because the biometric system was brought to its knees by Covid-19.
Fingerprint recognition lost its popularity because people are not allowed to touch surfaces while facial recognition also lost its prevalence because a mask prevents accuracy.
Facial recognition involves biometric identification that uses a person’s features to verify their identity. The technology collects a set of unique biometric data of each person’s face.
Msiza said that people think the technology can simply reconstruct the hidden facial features or remove the masks in some artificial way. The technology instead focuses on the features that are left exposed by the mask, such as the forehead and the eyes.
“What makes it difficult is that you are left with less than half of the facial landmarks exposed,” he said.
Msiza is the head of a technical service house that started working on this facial recognition prototype in May. He and his team do not have the finances to develop it further, and are now trying to sell it to original equipment manufacturers (OEMs) such as Samsung, Apple and Suprema.
“OEMs that we have approached are showing interest,” said Msiza.
According to a preliminary study by the National Institute of Standards and Technology released in July, identifying an individual wearing a mask was done “with great difficulty”.
The study found that algorithm accuracy with masked faces declined substantially across the board. Using unmasked images, the most accurate algorithms fail to authenticate a person, about 0.3% of the time.
Masked images raised these top algorithms’ failure rate to about 5%, while many otherwise competent algorithms failed between 20% to 50% of the time.
To get around this obstacle of the mask, Msiza used a hybrid solution that combines conventional image processing techniques and artificial intelligence methods.
He said that his prototype uses deep learning and machine learning in artificial intelligence. This kind of technology is capable of learning unsupervised data from unstructured or unlabelled material.
It uses algorithms inspired by the human brain to learn from large amounts of data. Essentially, just like humans learn from experience, the deep learning algorithm performs a task repeatedly, each time refining it a little to improve the outcome.
Tshegofatso Mathe is an Adamela Trust business reporter at the M&G