Deepfake
Home > 
Deepfake
< back

Deepfake

Introduction to Deepfake

Deepfake, a portmanteau of “deep learning” and “fake”, is the utilisation of artificial intelligence (AI) / deep learning algorithms to create believable or realistic videos, images and audios which never happen. An example of its application is to modify film footage instead of re-shooting and refining advanced face attributes in post-production.

In recent years, deepfakes have attracted public attention for their malicious uses in the creation of fake videos, forged images and financial fraud, resulting in the spread of misinformation or disinformation which can potentially erode the reputation of businesses and trust among people. Nowadays, tools that create deepfakes are becoming more readily available. Plausible deepfakes have elicited public responses to detect and limit their use.

How Deepfake Works?

Technology to create such synthetic data keeps evolving over time.

-
Manual Manipulation: Apply post-processing steps like scaling, rotating and colour adjustment to improve the visual appearance, scale and perspective coherence.
-
Automated Manipulation: Adopt digital computer graphics and deep learning techniques to modify videos, images or audios with better semantic consistency.
-
Artificial Intelligence (AI) Generation: Create convincing videos, images and audios from scratch using autoencoders or Generative Adversarial Network (GAN).
Evolution of Deepfake
Figure 1: Evolution of Deepfake

Deepfake technology includes a list of algorithms to recognise the most important parameters of human faces and voices, and learn to reconstruct them in synthetic videos, images and audios. The technologies are broadly categorised into visual and audio groups, and attributes manipulation levels are classified by source media and synthetic result is required.

The audio-based deepfake technique is used to create a convincing speech that sounds like given by a particular person who actually has not said so.

-
Text-To-Speech synthesis: Use a realistic narrator voice maker to convert text into natural-sounding speech.
-
Realistic AI voice: Use synthetic voice AI to render an authentic human voice from recording samples and communicate with others with great accuracy in tone and likeness.
-
Voice attribute editing: Modify voice attributes to modulate voice sound completely different.

The visual-based deepfake technique is used to create falsified videos which a person in an existing image or video is replaced with someone else’s likeness.

-
Face swapping: Replace the face of the person in the source video with the face of target person.
-
Lip syncing: Create a synthesising video of a target person such that the mouth movement in the manipulated video is consistent with arbitrary voice input.
-
Face re-enactment: Enable real-time rendering and interactive manipulation of the facial expressions of a target person onto a video.
-
Entire face synthesis: Generate non-existent but lifelike human faces.
-
Face attribute manipulation: Manipulate the facial appearance of a person by modifying the attribute-specific region while keeping other irrelevant regions unchanged.
Impacts of Deepfake

Since the quality of deepfakes is increasingly realistic and convincing, people can hardly distinguish fake attributes from a real video. Deepfakes seem so real that the viewers are led to believe that they are real and share them on social media channels thus hastening the spread of disinformation.

Many malicious deepfakes have been identified and some of the impacts of deepfakes are listed below.

Examples of Deepfake Impacts
Figure 2: Examples of Deepfake Impacts
1.
Creating negative effects of public opinion
Cause serious damage to victim's reputation such as misrepresenting certain celebrities using their faces or voices to spread rumours.
2.
Affecting economic activities
Depict make-up events to defame enterprises which may in turn drive the sentiments of the market and cause turmoil on stock market.
3.
Creating legal issues
Deepfakes may involve reproducing and distributing copyright material without owners’ permission and improper use may lead to legal problems. Similarly, disclosing or altering someone’ personal information may become subjects of privacy breaches. For litigation in court, dispute concerning electronic evidences such as audio or video recordings can be used only if its authenticity is testified by forensics watermarking tools.
4.
Facilitating cyber attacks
Deepfakes may have significant potential to enhance cyber attacks and scams.
-
Impersonate a trusted person and gain personal information for enhanced spear phishing attack (e.g. identity theft, CEO scam).
-
Use fake identity information to commit fraud such as “ghost fraud” of impersonating a deceased person for financial gain (e.g. fraudulent transaction).
-
Reduce the reliability of face recognition for liveness detection such as synthetic facial expression or cloned voice.
Combatting Deepfake

With the wide application of deepfake technology, it is increasingly difficult to believe what we see or hear. Thus, the detection of deepfakes is crucial to the protection of people’s trust in the digital content. Most of the work in detecting deepfakes make use of deep learning methods and exploiting visual artifacts present in the face regions. By recognising patterns in how deepfakes are created, it is able to pick up subtle inconsistencies using deep learning. Apart from using technologies, there are some general tips for detecting deepfakes manually:

General Tips for Detecting Deepfakes
Figure 3: General Tips for Detecting Deepfakes

Given the threats posed by deepfake technology carry serious and far-reaching implications for business, it is important to take preventive measures to reduce the chance of falling prey to deepfake-based frauds.

Best Practices for Organisations
Figure 4: Best Practices for Organisations
1.
Prevent
-
Encourage fact-checking processes in information handling.
-
Establish internal procedures to authentic suspicious contents, obtain its origins and identify its creator.
-
Maintain consistent information distribution / dissemination and reporting channels.
-
Provide proper training to employees on detecting reality of images and videos.
2.
Detect
-
Monitor business-related online media frequently to filter out any fake images or videos.
-
Perform comparative analysis to compare authentic content from media library or reverse image search on the Internet.
3.
Protect
-
Implement digital watermarking in the shared contents to avoid them being reused in generating deepfakes.
-
Deploy anti-spoofing tool for facial biometrics to provide liveness detection and facial recognition against deepfakes.
-
Deploy media authentication and provenance tools to verify the authenticity of the media.
4.
Response
-
Develop an incident response plan to contain and take actions when disinformation arises.
-
Identify the misinformation / disinformation and respond misleading statement immediately.
Extended Readings
1.
Hong Kong Computer Emergency Response Team Coordination Centre – Adopt Good Cyber Security Practices to Make AI Your Friends not Foes
2.
The United States Department of Homeland Security (DHS) - Increasing Threat of DEEPFAKE Identities
3.
Cyber Defender - What is Deepfake?

Disclaimer: Users are also recommended to observe the disclaimer of this website and read the user agreements and privacy policies of the security software and tools before downloading and using them.