AI Violence: Thick Fakes and the Need for Grounded Policies

In January 2025, at the peak of campaigning for the federal elections, a far-right political party in Europe posted a video on Twitter (X). The two-minute-long video contained AI-manipulated visuals with no disclaimers for AI use.

The scenario in the video is dystopian. It opens with an impoverished old man scouting for used bottles to earn a cent by feeding them into a recycling machine—a common scene in Germany known colloquially as collecting the “deposit bottles” (Pfandflaschen). Next, we see a swift flow of passengers and closeups of two young white passengers desperately waiting at a train station for the trains, but the tracks, alas, are empty. Right then, as more desperate faces become visible in the frame, a young Black man is seen harassing a white woman on the platform, leaning over her and lifting her shirt hungrily. The woman turns away—her horrified face staring into the camera.

The scene shifts to daycare centers and then to depictions of left-of-center politicians, signaling a future Germany crippled with economic misery. The trope of Black criminality recurs. A young Black man is now seen at a public park, looking wily and dealing drugs. The word “remigration”—the right-wing slogan to deport “non-ethnic” Germans—flashes on the screen. A group of immigrants is seen dragging their suitcases in the direction of a deportation flight. As the aircraft takes off, a Black male face stares through the flight window helplessly, and the lush green expanse below comes into view. The green wonderland is seized from the Black immigrant, which he can never aspire to regain, and freed for eternal peace for the “natives.” The video ends with a direct appeal to vote for the party and happy scenes of promised prosperity for the country.

The far-right party’s campaign video is among the thousands of instances of generative AI’s expanding presence in the political field. News and policy commentaries often pitch this as a crisis of Truth, with a capital T. “Deepfakes”, as the diagnosis goes, can break down the pact between image and reality, and can thereby endanger democracy.¹

The anti-immigrant video, however, does not operate on grounds of truth and falsity. It unfolds as conjecture and anticipation. It is about what could happen and not a claim about what has happened. Such videos are not deep fakes that deceive, but “thick fakes” that are absorbed as socially plausible.

Thick fakes and thin fakes

“Deepfakes” are indeed a misnomer, or the very least, misleading. “Deepfakes” that takes its name from deep neural networks of AI are not socially deep in and of themselves. In place of the broad moniker of “deepfakes”, it is useful to draw a distinction between “thin fakes” and “thick fakes”.

Thin fakes are socially frivolous and fragile even when they feature advanced AI tools. They are easily discerned and even actively consumed as fakes. They are often framed as play and humor. In contrast, thick fakes draw upon entrenched narratives and social rhythms of gossip. They are thick because they resonate with felt social worlds as conceivable.

Under this main distinction between thin and thick fakes, there is substantial variation in the social registers of fakes that are mobilized for electoral politics globally, including voice clones that are deceptive, celebrity fakes that rely on stardom as an engine for electoral messaging, and the grotesque and absurd images of AI slop that can both perforate and elevate political grandstanding. AI-generated far-right videos constitute a significant type. They represent new forms of aestheticized exclusion that enact the brutal violence of visuals by materializing desired or feared futures.

Synthetic scenarios

The anti-immigrant video is a stark example. With the use of AI technologies, the video presented synthetic bodies and backdrops as the testimony of Black criminality and the cause for society being brought to the brink. These technologies helped the party to simulate a realistic scenario—moving through the exact mundane activities that people are most familiar with—and infusing scenes of bottle collection, people waiting for the train, children tumbling around at the daycare and so on, with searing panic and racist fear. Characters can now be conjured instantaneously with generative AI, and myriad racist portrayals can flow through seamlessly. While the characters portrayed exactly what the creators wanted, creators on their part did not have to bother about what the characters would have actually said and conveyed. The video creation posed no friction that a real world with real people would present, while every motif in the video drew upon scenes near and familiar, presenting the scenario as now or the not-too-distant future.

Alongside dystopian videos, the far-right party posted AI-powered images of heteronormative white families happily grilling bratwurst (sausage) by the riverside. Such depictions seek to recover the glorious past that is imagined to be lost but undead. The sociologist Zygmunt Bauman calls this genre “retrotopia.”² These fakes do not resort to an explicit language of hatred. They work through aestheticized erasures of heterogeneous milieus. By building up idealized scenarios, they conjure a restored white normative future and narrate its imminence. The AI-generated visuals of “Gaza Riviera” posted by US President Donald Trump are an example of how such fakes need not even have any reference to the imagined past but just brazen ideas of a fantastical future.³ The glossy AI-generated fantasy of luxury skyscrapers overwrites violent annihilation needed for its realization.

In such instances, the imagined past or the imaginary future appears more material via synthetic characters and scenarios, “giving the imaginary a ‘real-like’ quality that makes it strictly more than merely imagined.”⁴ The “concreteness” that AI provides to scenarios can normalize new thresholds for hate.

Labeling fakes is not enough

The synthetic violence of thick fakes cannot be addressed through labeling, the most common regulatory instrument currently used in tackling AI fakes. The kind of fakes typified by far-right anti-immigrant videos appeals to sentiments, identities, and entrenched fears and conceits. On the one hand, as synthetic scenes, they evade regulation because they do not falsely depict any natural person or real world events. On the other hand, their appeal can be stronger than any false depictions of the real world. Their traction is linked to social structures that are hard-boiled and longstanding.

The emotional appeal of synthetic imaginaries is by far the hardest challenge for policy and regulation. Policy efforts should stay sensitive to how people engage with AI media and develop measures from the grounds up.

An important intervention is to develop a professional code of conduct and co-regulatory models for political consultants who offer election campaign services to political parties. These measures should be complemented with regulations for campaign finance and election expenditure; restrictions on platforms that host AI fakes; limits on technology use for political campaigning, including clear guidelines for political parties and politicians; training and awareness raising among influencers and communities; and resource allocation for timely factchecks that do not merely offer corrections to false claims but also critically intervene by delegitimizing synthetic scenarios and their narrative hold.

Acknowledgments:

This article draws on the author’s upcoming book, Divided by Machines: Extreme Speech, Deception and the Limits of Artificial Intelligence, New York University Press.

Sahana Udupa is Professor of Media Anthropology at LMU Munich and BKC Faculty Associate at Harvard University.