Despite the damage to both celebrities and the victims who click on these fake adverts, which generate revenue for companies such as Meta, it is not easy to get them removed from Facebook.
(Photo by Chesnot/Getty Images)
South African celebrities are increasingly the target of deepfake adverts but social media platforms such as Meta’s Facebook appear to be not doing enough to remove these high-revenue generating posts.
Digital advisory firm Liminal predicts that the cost of deepfake detection to combat this rising threat is set to grow from $5.5 billion last year to $15.7 billion by 2026, at a compound annual growth rate of 42%.
According to digital security company Trustfall, there was a marked global rise in cybercrime last year and in the first half of this year, with the biggest increase noted in deepfake fraud and super-synthetic identities.
In South Africa, victims of the adverts, which use digital manipulations generated by advanced artificial intelligence (AI), range from business people such as Johann Rupert, Nicky Oppenheimer, Patrice Motsepe and Elon Musk — purportedly promoting online trading platforms, in which consumers have lost millions of rands — to TV presenter Leanne Manas, musicians Francois van Coke and Karen Zoid, as well as sports scientist Tim Noakes.
Their identities have been stolen to promote everything from online trading platforms to diet “keto gummies” and pills for erectile dysfunction.
The Noakes Foundation has commissioned research into the growing problem and has set up a website where people can report fake endorsement adverts.
“Meta will be taking a large chunk of that in advertising-spend revenue to distribute this fake malice content.
“It’s hard not to draw the conclusion that it seems irrelevant to Meta if the content is genuine or a scam or if the account used to promote these scams has been hacked or cloned.
“Either way, Meta and Facebook profits from these scams,” said the foundation’s director, Travis Noakes, son of Tim Noakes.
The “scams’ prevalence has ramped up dramatically”, Noakes said.
By 2015, only a handful of local celebrities had been brand-jacked. These included Minki van der Westhuizen, Jeannie D, Basetsana Kumalo, Nkhensani Nkosi and Shashi Naidoo.
“There were nine reports of SA micro-celebrities being brand-jacked in 2023, with 11 so far in 2024.”
Among these werwe Manas, Oppenheimer, Motsepe, Noakes, Van Coke, Zoid, Dr Shabir Madhi, Dawn Thandeka King, Anele Mdoda, Karlien van Jaarsveld, Naas Botha, Trevor Noah, Professor Kgomotso Mathabe, Dr Santosh Bugwandin, Jan Hendrik van der Westhuizen, Steve Hofmeyr, Anton Rupert, Helen Zille and Francis Herd.
“The exact extent of the problem is unknown in South Africa and there is also reluctance from celebrities to talk about its impact on them personally.
“However, we can see from our own experience with Prof Tim Noakes and the ‘keto gummies’ scam that people have lost hundreds of thousands of rands to the scammers,” Noakes said.
“Where products are delivered, people are in danger of consuming a fake and potentially harmful product. In addition, once people engage with fake adverts, they are at risk of having malware installed on their devices.”
Scammers use malware — software designed to disrupt, damage or gain unauthorised access to a computer system — to get credit card details and home or office addresses.
“As a consumer, you not only lose out on a fake product that often won’t make it to your door, you are also putting yourself at serious risk of credit card fraud,” Noakes said.
The Mail & Guardian has recently reported on cases where investors have lost money after clicking on deepfake adverts using celebrities promoting online trading platforms that have led to individual losses ranging from tens of thousands of rands to more to R1.7 million.
One of the investors the M&G spoke to believed he was speaking to Elon Musk on Facebook Messenger and that he was investing in his new online platform.
Celebrities suffer from stress, the embarrassment of being associated with a digital crime, harm to their reputation and possible loss of “opportunities”, Noakes said.
“For example, a public health expert associated with endorsing ‘diet pills’ could well be ignored for serious public speaking engagements.
“A third impact is scarce resources must be spent fighting a digital crime, for example, in paying for legal correspondence to social media platforms. These resources could be used for better purposes, such as public health promotions,” he said.
Despite the damage to both celebrities and the victims who click on these fake adverts, which generate revenue for companies such as Meta, it is not easy to get them removed from Facebook.
“In interviewing three celebrities and/or their representatives we learnt that, in every case, Meta was very slow to remove content.
“Automated reports of celebrities’ brand-jacking were largely ignored and the only sure way to ensure a robust response was to report fake accounts and related content to a third party that could report the problem to Facebook South Africa,” Noakes said.
But most brand-jacked celebrities do not enjoy this support and, by the time Facebook responds, the financial damage has been done, he said.
“Facebook is effectively facilitating digital crime by not taking prompt action in response to legitimate reports of fraud.
“Worse, Meta’s advertising service benefits from the proceeds of crime in being paid for hosting these ‘ads’.
“In addition, Facebook may further reward cybercriminals’ who’ve hacked business advertising accounts with royalties for clicks from their ‘successful campaign’.”
Noakes explained that, using resources available on its website, “we gift-wrapped spreadsheets of fake accounts and advert addresses for Meta”.
“We laid them at their door time and time again, only to be told that these accounts did not go against their community standards.
“Even Meta’s ‘verified account’ status is a false sense of security and we have quickly learnt that this carries no impact in assisting with taking down the deepfakes,” Noakes said.
“We are yet to be heard through the traditional reporting system.”
Meta’s Facebook reporting also does not work for digital crimes that can potentially span multiple frauds.
Brand-jacking through fake celebrity endorsement spans several crimes: impersonation, non-consensual image sharing and the infringement of a public figure’s intellectual property through copyright violation of still images and audio-video.
Facebook typically supports reporting only one type of infringement at a time, potentially leaving a blind spot based on users reporting differing aspects of the celebrity brandjacking, Noakes said.
“For the ‘Dr Noakes erection pills’ advert, we made multiple, almost 100 in total, reports from the ‘verified account’,” Noakes said.
Facebook’s response remained the same: “Thanks again for your report. This information helps us reduce unwanted content for you and others. We use a combination of technology and human reviewers to process reports and identify content that goes against our Community Standards.
“In this case, we did not remove the content that you reported. If you think that we’ve made a mistake, you can request a review of this decision within 180 days.
“We understand that this might be upsetting. If you want to see less of Brentlinger on Facebook, you can unfollow or block them.”
“Reaching beyond Meta’s AI tools and bots was the only way for us to get this resolved, and this was only through a stroke of luck, with a resource that is not accessible or available to the general public,” Noakes said.
“It was disturbingly obvious that Facebook’s claim to ‘use a combination of technology and human reviewers to process reports and identify content that goes against our Community Standards’ was simply not true or certainly the AI in place to manage the reporting system is not capable of recognising the obvious issues.
“If this content had truly been verified by a human it would have been taken down immediately. Even to the most untrained eye it was obvious this content was a deepfake.
“Despite reporting a fake video more than a 100 times the content still went viral and achieved over half a million views,” he said.
He accused Facebook of “turning a blind eye to this digital advertising crime”, adding: “The benefit to Meta is clear with them reaping the rewards through advertising spend.”
Manas previously told the M&G she had struggled to get fake accounts and posts using her stolen identity removed from social media platforms, including Facebook.
A Meta spokesperson said scammers used every avenue available to them to defraud people and adapt to evade enforcement.
“Content that purposefully intends to deceive or exploit others for money violates our policies and we remove violating this content when it’s found.
“We continue to invest in detection technology, review teams and we share information with law enforcement so they can prosecute scammers. We also actively share tips on how people can protect themselves, their accounts and avoid scams.”He asked that the M&G forward the links Noakes previously shared with Meta for further review.