Synthetic media: Deepfakes

“A deepfake is meant to cause an incongruence between expectation and perceived reality.” – Dobber et al.

On deepfakes of public death, or “digital desecration”: “Public death has long been a site of spectacle and power, and mockery of the dead predates the internet. [What’s new with AI deepfakes] … is the medium’s claim to presence. AI impersonation [of the dead] shifts desecration from depicting someone to ventriloquizing them: synthesizing face, voice, and affect and then staging ‘new’ actions, lines, and reactions. That’s not just cruelty; it’s an authored performance that borrows the authority of the person’s likeness. AI also enables digital desecration at greater scale and immediacy.” – Julie Carpenter, Ph.D., author, “The Naked Android: Synthetic Socialness and the Human Gaze.”

Overview

This chapter is a continuation of last chapter’s exploration of purely AI-generated content. Deepfakes, however, combine existing real content with AI-generated media to create new content that appears to be representing a real person’s communication.

Here is a Bloomberg Quicktake that demonstrates what this looks like.

Naturally, there are concerns beyond using deepfakes as a form of entertainment. If deepfake technologies were used exclusively by Hollywood film effects specialists, perhaps these clips wouldn’t be too much cause for concern. (Do you recall how impressive it was in 1994 when Forrest Gump met President Kennedy and Kennedy’s speech was synthesized?).

However, deepfake software is freely available today and nearly anyone with a desire to make them can do so at minimal cost.

What does it mean when a single person can produce fake communication and impose asymmetrical chaos into our social and political well-being?

Below: A deepfake video experiment produced by Ethan Mollick, a leading researcher and writer in the area of AI and LLMs. Note that the clip includes AI-generated voice and picture in multiple languages.

Key Terms

The Liar’s Dividend: In the article “Deep Fakes: A Looming Challenge for Privacy, Democracy, and National Security,” the authors, Robert Chesney and Danielle Keats Citron, propose that the risk of deepfake media permeating society is not so much in the misleading perception posed by actual deepfake content but, instead, in the notion that even legitimate media could be synthetic. The “dividend” contained in the phrase refers to the benefit gained by powerful entities or individuals when the public does not know for certain whether any media is real, particularly when an individual denies culpability for behaviors recorded on video by claiming that the video was a fake.

Read more about the research conducted to measure the impact of deepfake media on human perception.

There are two examples worth exploring that demonstrate the undermining effect of deepfake videos on the perception of actual, non-fake video content:

• “GOP House candidate publishes 23-page report claiming George Floyd death was deepfake video” – A conspiracy claim that the George Floyd videos depicting his apprehension and death by police was faked.
• “A Military Coup in Gabon Inspired by a Potential Deepfake Video is Our Political Future” – The somewhat odd appearance of president Bongo of Gabon in a video address sparked a controversy by his opponents claiming that the video was a deepfake. The video was, in fact, authentic, though the president had experienced a stroke which altered his appearance.

What should you be focusing on?

Your goal this module is to assess the influence of deepfake media in the human effort to form reliable mental models of the real world. As you review the readings and media, think about what kind of problem this is from the perspective of your chosen framework.

Readings & Media

    Experiment     Test yourself!

Go to Detect Fakes on the MIT website and test your ability to detect a deepfake video.

After testing yourself, review the features of Reality Defender, a deepfake detection system.

    Required     “Deepfakes, explained” by Meredith Somers, MIT Sloan School of Management, July 21, 2020 (9 pages)

This article provides a foundational explanation of deepfake videos with links to some prominent examples. Be sure to view the deepfakes of Mark Zuckerberg and Kim Kardashian.

    Required     “Animatable Gaussians: Learning Pose-dependent Gaussian Maps for High-fidelity Human Avatar Modeling” by Zhe Li, Zerong Zheng, Lizhen Wang, Yebin Liu1, November, 2023.

Scroll to the bottom to review examples of extrapolated movement mapped to lifelike avatars. The quality of these examples is outstanding though there are still discernibly synthetic features. Consider the potential to generate lifelike avatars of targeted individuals performing actions that they did not do. Pay close attention to the realistic movement of the clothing as the avatars move – a feature you might not even notice unless directed to look for it.

Below is a variation of this technology demonstrated by taking a still reference image and mapping it onto existing biometric-based human movement. (This video has no sound).

    Required     “Deepfakes 2.0: The New Era of ‘Truth Decay’” by Brig. Gen. R. Patrick Huston and Lt. Col. M. Eric Bahm, Just Security, April 14, 2020 (4 pages)

This article describes the scope of the misinformation problem and offers some solutions. An excerpt of the potential effects:

• Fake Evidence: Manipulated videos being used as evidence in court.
• Sparking a war: a fake video of Israeli soldiers physically assaulting a Palestinian child could spark a new wave of violence in Israel.
• Manipulating Markets: fake videos of a CEO used to disrupt an initial public offering.
• Creating Political Fissures: fake videos intended to sow discord between foreign allies.
• Influencing Elections: A doctored video of a politician looking sick designed to tip the scales of an election.

Related: Nikon, Sony and Canon fight AI fakes with new camera tech (1 page)

Related: Deepfake deluge expected from AI image generation breakthrough (so long, LoRA?) (1 page)

    Required     “Is seeing still believing? The deepfake challenge to truth in politics”  by William A. Galston, Brooking Institute, January 8, 2020. (8 pages)

This article touches upon several of the topics we have covered in this course: engagement with mediated communication, epistemological differences, and subjectively optimized information bubbles.

Look for the author’s framing of the problem and how he approaches combating it. Think about how you would approach the problem according to your chosen framework.

For further study: “Deepfakes: The Coming Infocalypse” by Nina Schick.

Optional: “Nvidia researchers debut GauGAN, AI that creates fake landscapes that look real”

Optional: Examples of AI-generated audio samples imitating known speakers.