A day after Delhi Chief Minister Arvind Kejriwal was arrested by the Enforcement Directorate in a liquor scam case, his wife Sunita read out a message from him on camera, in a video released by the Aam Aadmi Party online. But that was just one version of Kejriwal’s message that went viral — an Artificial Intelligence-generated English translation of the incarcerated politician’s communication followed, and then another Hindi version. “I’m neither shocked nor worried; all my life, I’ve struggled for a better society,” these videos said, in Kejriwal’s voice.
Also read: Author and podcaster Anirudh Suri explores the #AIforGood formula
As India gets into election season, more and more examples of synthetic and realistic deepfakes of politicians have started appearing. There’s one of the late M. Karunanidhi exhorting party cadre, as his son, Tamil Nadu Chief Minister M.K. Stalin, looks on; a Tamil dub of Prime Minister Narendra Modi addressing a gathering in Chennai; and videos of Madhya Pradesh leaders Shivraj Singh Chouhan and Kamal Nath with doctored remarks. Anxiety about AI deepfakes in campaigning has heightened largely because making synthetic images and videos has gotten far cheaper — and better — from four years ago, when the Bharatiya Janata Party’s Manoj Tiwari put out a low-resolution AI-generated Haryanvi dub of a video message in the run-up to the Delhi Assembly elections.
Also read: Regulating deepfakes and generative AI in India | Explained
India’s neighbours too have lent some legitimacy to these anxieties: voter suppression deepfakes with a “don’t show up; the polls are rigged against us” theme have surfaced in the past year in both Pakistan and Bangladesh, where the targeted opposition politicians had to issue denials. With cheap mobile data and the highest smartphone penetration in India’s history, concerns are rife that similar crucially timed deepfakes circulated via platforms such as WhatsApp could sour voters on candidates, or convince them that their vote could be meaningless.
Government issues advisory
‘Ethical’ AI creations
Not all synthetic AI creations take the form of sinister opponent slandering. Some implementations, like personalised interactive phone calls, seem more like interesting novelties than threats to electoral integrity. While one-to-one calls are not yet a reality in India, so-called ‘blasters’ with a synthesised candidate’s voice speaking individual voters’ names in a pre-recorded message have been sent out by the Congress party in Rajasthan and by the Aam Aadmi Party in Delhi.
Divyendra Singh Jadoun, one of an emerging crop of synthetic media creators, has assembled half-a-dozen staff to train voice and video AI models and distribute them through phone calls and video messages, on behalf of political parties. While Jadoun, who operates under the name The Indian Deepfaker, refused to name the organisations, he said that at least four of his current projects are on behalf of political parties, with at least two mainstream organisations in the fray.
Jadoun says he restricts his firm’s work to “ethical” AI creations such as authorised translations, revivals of deceased leaders (with the party’s blessings), and one-on-one phone calls with chatbots synthesising tailored responses. He uses Mistral AI, which is based on open source models, to get around mainstream proprietary firms’ refusal to allow their tools to be used in electioneering.
He claims to have refused unethical requests by parties to depict opponents saying things they never have; but notes that increasingly, parties prefer to not outsource deepfakes of rivals to external firms, instead taking the task upon themselves. “Anyone with a laptop now can make this stuff,” said Karen Rebelo, deputy editor of the fact checking website BOOM, at a panel discussion this February. “You don’t need to go to a specialised agency, or even to somebody who knows code.” Rebelo noted a sharp increase in AI deepfakes already during recent Assembly elections in India.
Sagar Vishnoi who worked at The Ideaz Factory, the firm that made Tiwari’s Haryanvi video in 2020, agrees that the technology has become much cheaper. Tiwari’s lip sync back then was done over a day-and-a-half, and his voice was not synthetic — a mimicry artist had dubbed over the MP’s video. “Now, the tech has changed. Voice training models have become available, not just lip syncing ones,” says Vishnoi, who has since left the company and is currently engaged as a political consultant for a clientele that he declined to name.
Tackling misinformation
But even those wary of the fairness of Indian elections in recent times are not entirely convinced that deepfakes will affect the integrity of the poll process any more than the conventional strategies already at play. Says Pratik Sinha, a co-founder of fact-checking news website Alt News: “AI is an issue, but I don’t feel as bothered about a new way of creating disinformation; the existing methods are working quite well.”
Sinha, whose daily job involves flagging fake news and misinformation, is referring to the strategically clipped videos and inflammatory speeches that continue to be circulated by the dozen from ordinary citizens’ smartphones. “What has changed from 2019 to now is that the amount of hate speech has increased manifold,” he says.
Abbin Theepura, a veteran political consultant who has worked extensively with social media, says that AI technologies are still “nascent”. More than synthetic media, the challenge today is the automating of distribution of content through micro-targeting, something tech firms have already been working on perfecting for a decade. “It’s not like AI is helping me automate the content,” says Theepura. “That still needs to be generated in the first place.”
Political observers are, however, looking closely at what the BJP, in particular, might get up to, for a simple reason: the party’s early lead on using technology in past elections. The BJP’s well-documented and extensive use of social media was underway even in the 2014 elections, before smartphone penetration was anywhere near the levels today. Opposition parties have had to play catch-up in the following years. If nothing else, what kind of experiments the ruling party runs will be useful in assessing what techniques it adds to its armoury.
Translation into multiple regional languages is obviously a big use case: while the Prime Minister has used the government’s Bhashini app to do live translations of his speech at a meeting last year, real-time speech recognition in Indian languages is not perfect, and neither is translation technology. Vetted translations synthesised into a dub track that are distributed much after a speech has been delivered can get the message across fairly cheaply — without ringing any alarm bells a la deepfakes. Jadoun says that translations are typically put through a quality analysis and manually verified before being fed into the software.
What could ultimately determine the level of impact of any sort of AI-enabled campaigning tool, though, is distribution — another strength buoying the BJP with its large presence in States, offline as well as on social media. Opposition parties are at a financial disadvantage here, and they argue, suffer from retaliation that the ruling party does not. “Technology like AI conversation [phone] calls will become very common as more vendors start making them available to political parties,” says Vishnoi. “If you already have a distribution network created, it becomes very easy” to dominate the field. “The BJP has a strong and structured distribution network,” as do some other dominant regional parties, he adds. Vishnoi cites Tamil Nadu’s ruling Dravida Munnetra Kazhagam and its rival, the All India Anna Dravida Munnetra Kazhagam, as organisations that have garnered an early lead.
But opposition parties continue to remain wary of the reach and result of their messaging in the face of widespread misinformation and fake news. “I can share a list of YouTube channels that are only peddling fake news day in and day out and that [are] retweeted by members of a certain political party,” says Jasmine Shah, an Aam Aadmi Party appointee chairing the Delhi government’s Dialogue and Development Commission. “But obviously no action” is taken against these. He compares this to when Delhi Chief Minister Arvind Kejriwal got slapped with a defamation notice in 2019 after he retweeted a video by YouTuber Dhruv Rathee about the BJP’s IT cell. (Shah spoke in February, a month before Kejriwal was arrested by the Enforcement Directorate.)
In other words, the playing field is uneven if a party decides to play dirty. Vishnoi says it would be ideal if all political organisations came together to declare a common list of principles on AI use, such as labelling of synthetic content and avoiding depictions of opponents.
Before things get there, though, parties appear poised to see what this technology can or cannot do in furthering their ultimate objective: winning the election. “Whatever tools are coming, we will experiment,” says Theepura. Whether or not they have an impact, the AI techniques deployed this election season may well show us a glimpse of what digital campaigning will look like in the coming years.
aroon.deep@thehindu.co.in
(With inputs from Neha Mehrotra)
Special digital series
This election season, misinformation has a new face. While the 2019 elections were no stranger to hate speech and disinformation campaigns, the technology that enables this ecosystem has revolutionised at warp speed. In our ongoing digital series, we at The Hindu decode how the nature of election-related misinformation online has evolved, from social media bots to deepfakes. The articles will include interviews with legal experts and activists who will throw more light on the following topics:
- What regulations did and didn’t work in the pre-generative AI era, circa 2019
- How algorithmic and social media design changes in the last five years have fuelled the misinformation landscape
- Why social media platforms such as X, Meta and YouTube are struggling to contain fake news and propaganda
- The emerging market and technology of deepfake makers and their political ‘content’
- What tech companies are doing to combat AI-generated misinformation
- What effect AI can have on the global electoral landscape
The series will be a handy toolkit for the Indian voter; a guide that attempts to make sense of the synthetic chaos that undermines trust and truth in a democracy. Click this link to read more.