Issue 6-Generative AI Scams on the Rise: What Does the Future Hold?

Picture this: You get a call from your CEO, urgently requesting a transfer of funds. Everything sounds legit—it’s their voice, their tone. But it’s not them. It’s a scammer using deepfake technology, and you’re caught in their web. Scary, right? 

On the other hand, it could be a fraudster completely generating realist images or IDs of a high-networth individual and using them to bypass KYC checks and access their accounts. These scenarios are becoming all too common across Africa.

Over the past year, the availability of user-friendly deepfake applications has skyrocketed. These tools, often available for free or at a low cost, enable criminals with minimal technical expertise to create convincing fake content.

Cybercriminals are increasingly using deepfakes to impersonate CEOs and other high-ranking officials in "CEO fraud" schemes. In one notable case, Fraudsters used AI to impersonate African Union Chief Moussa Faki, faked his voice and placed several video calls to European capitals, ostensibly seeking to arrange meetings. 

The African Union Commission revealed that the cybercriminals used fake email addresses to impersonate the organisation’s deputy chief of staff. These criminals arranged calls between foreign leaders and the AU Commission Chairperson, Mr. Moussa Faki. Spokesperson Ebba Kalondo confirmed that the pranksters conducted video calls with several European leaders using deepfake technology to mimic Mr. Faki.

Imagine how much easier that would be with the level of advancement deep fake technology has experienced within the last 1 year.

We’re All in Trouble, What Next? 

The big question becomes, how we protect against deep fakes? To answer this accurately, let's look at how deepfakes are created. 

• Data Collection: The AI algorithm is fed a large dataset of video or audio clips of the person to be mimicked.

• Training the Model: The AI learns the person's facial expressions, voice patterns, and mannerisms.

• Generating the Deepfake: Using this trained model, the AI can produce fake video or audio that appears authentic.

The nature in which generative AI content is created makes biometric-based solutions the most effective in combating them. Enhanced biometric verification methods, such as facial recognition and active liveness detection are crucial in the fight against deepfakes. Liveness tests provide better security with the ability to catch synthetic media and deepfakes.

Active liveness refers to specific tasks that the user is asked to perform (often part of a larger liveness system), whereby the recorded sequence of images (video) is then analysed to verify that the task was indeed correctly performed. If the user correctly performs the tasks, then the user has passed the 'active liveness' challenge and further verification checks can be performed (for example spoof detection). This solution leverages the user's biometrics.

AI voice detector could also be used to determine the source of an audio file, whether human or AI-generated.

Although technology plays a key role in combating these scams, businesses need to establish compliance policies that address the growing risk of generative AIs and train employees on how to approach different scenarios. 

Ultimately, combating generative AI deep fakes requires a blend of the right technological solutions and expertly trained humans. 

Fraud Watch

• Interpol Operation Nabs 300 with links to West African Cyber Fraud: Interpol announced on Tuesday its arrest of 300 individuals and seizure of $3 million in assets. Read more here.

• Specific Institutions Urged to Note Heightened Terror-Financing Risk in South Africa: To ensure greater awareness of the nature of the risks that exist, South Africa has recently completed its second Terror Financing National Risk Assessment (TF NRA 2024), alerting affected institutions on the higher terror-financing risk. Read more here.

Country News Coverage

• SARB Sanctions Giant Insurance Company for FICA non-compliance: This month, the South African Reserve Bank (SARB) announced that it had imposed an effective fine of R7 million on Safrican Insurance Company following a FICA compliance inspection. Read more here.

• SA’s Property and Legal Practitioners Keeping Country on Greylist: The Financial Intelligence Centre (FIC) says in a statement that it identified business sectors wilfully standing in the way of South Africa exiting the greylist and legal practitioners top of the list. Read more here

Ask the Experts

We have no questions for today and look forward to your submissions. Hit reply and let's chat on anything identity verification or AML compliance related in Africa.  

Till next time,

The Smile ID Team.