Press "Enter" to skip to content

Perception won’t be reality, once AI can manipulate what we see

Voice-spoofing technology was used to steal a quarter-million dollars in March from the unwitting CEO of an energy company, who thought he was talking to his (German) boss. A recent study showed that 72 percent of people reading an AI-generated news story thought it was credible. In September, a smartphone app called Zao became a viral sensation in China; before the government abruptly outlawed it, Zao allowed people to seamlessly swap themselves into famous movie scenes.

Then there is that infamous case of doctored video of the House Speaker Nancy Pelosi (D-Calif.) that went viral before being detected as being manipulated to make her appear drunk.

Most of the recent advances in AI — artificial intelligence — have come in the realm of perceptual intelligence. This has enabled our devices to see (and recognize faces of our friends, for example), to hear (and recognize that song) and even to parse text (and recognize the rough intent of the email in your mailbox). Today’s AI technology can also generate these percepts — our devices can generate scenes and faces that never existed, clone voice to generate speech, and even write pithy (if stilted) responses to the emails in your inbox.

This ability to generate perceptions puts AI in a position of great promise and great peril.

Synthetic media can have many beneficial applications. After all, inducing suspension of disbelief in the audience is the cornerstone of much of entertainment. Nevertheless, it is the potential misuses of the technology — especially going under the name of “deep fakes” — that are raising alarms.

If perception is reality, then what happens to reality when AI can generate or manipulate perceptions? Although forgeries, fakes and spoofs have existed for much of human history, they had to be crafted manually … until now. The advent of perceptual AI technology has considerably reduced the effort needed to generate convincing fakes. As we saw, the Zao app allowed lay users to swap themselves into movie scenes. What is more, as the technology advances, it will become harder to spot the fakes. Sites such as  “Which Face is Real?show that, already, most people cannot tell AI-generated images from real ones.

Easy generation and widespread dissemination of synthetic media can have quite significant adverse consequences for many aspects of civil society. Elections can be manipulated through spread of deep fake videos that put certain candidates in compromising positions. Spoofing voice and video calls can unleash a slew of new consumer scams. Individual privacy can be invaded by inserting people’s likenesses into compromising (and sometimes pornographic) pictures and videos.