Deepfake sockpuppets: the toxic ‘realities’ of a weaponized internet.

Deepfakes combine the latest in machine learning and Artificial Intelligence to create hyper-realistic audio-visual forgeries. Using state of the art technologies to engineer computer-generated, synthetic movement synchronised with fake audio tracks, deepfakes represent the next generation of digital media manipulation and disinformation: online propaganda cognitively more arresting than other text or image-based fake news, and which improve in quality with each new fake.

Industry experts and academics in the computer sciences have acknowledged the use of deepfakes to produce pornographic videos, fraud and fake news. This chapter investigates the pervasive, deceptive potential of deepfake ‘sockpuppets’, that is the creation of fake identities that generate and perpetuate toxic propaganda online. These ‘sockpuppets’ operate undetected as internet users do not have the forensic technology nor media literacy to identify the forgery. Even global news agencies and their editors do not have the expertise of global intelligence agencies who might be able to detect algorithmic glitches or weaknesses. This has meant that these non-existent, untraceable personae have manifested as international journalists generating publishable content, with the intention of distorting public perception and discourse. Perhaps more worrying, is that even on the discovery of the deepfake this ‘news’ is not retracted nor removed from websites.

This chapter investigates who is behind the ‘sockpuppet’ and their intentions. It argues that they are deployed as the cutting edge in toxic media propaganda to undermine cultural and political relationships and campaigns. This form of deepfake is purposefully provocative, working to compromise world leaders and remind nations of geopolitical conflicts. Through the use of xenophobic, racist and misogynist ideologies ‘sockpuppets’ incite social tensions whilst also undermining the internet as a source of reliable factual information. This has the startling potential to weaponize the internet, moving to a position where nothing is ‘true’ and maybe that doesn’t matter.