The story so far: 2024 is the biggest year for democracy in election history. Half of the world’s population will head to polls this year, and almost a fourth of them live in India. While India will be the nucleus of democracy’s biggest litmus test, the strength of its democracy is under scrutiny. V-Dem Institute’s recent democracy index termed India “one of the worst autocratisers”— it had already ceased to be a democracy on this index in 2018. Similar indices have downgraded India’s democratic standing in recent years: India is only ‘partly free’ (Freedom House), is home to a “flawed democracy” (The Economist Intelligence Unit) and is better classified as an “electoral autocracy.”
The Indian Government has refuted these assessments. It now plans to release its own democracy index, which, according to an Al Jazeera report, will help India “counter recent downgrades in ratings and severe criticisms by international groups”.
Both democracy and its global estimations are under challenge in India. The Hindu breaks down the makings of a global ‘democracy index’: how democracy is defined, what parameters are picked, which methodology is employed and why there can be no one perfect measure for democracy.
But first: why does India care about a democracy index?
From the Varieties of Democracy (V-Dem) project to Freedom House, there is a consensus that India’s democracy is in peril, comparable to conditions in 1975, when then Prime Minister Indira Gandhi declared a state of Emergency. Ahead of the election season, these indices and “negative commentary” by think tanks and agencies threaten India’s sovereign ratings and its ranking on the World Bank’s Worldwide Governance Indicators, the Al Jazeera report noted.
India has previously denounced all global rating assessments of Indian conditions, from democracy and press freedom to hunger, human development and happiness. Foreign Minister S. Jaishankar in 2021 called the makers of these indices “self-appointed custodians,” who are “not motive-less.”
“They invent their rules, their parameters, they pass their judgments and then make out as though this is some kind of global exercise,” said Mr. Jaishankar, implying there is a “very political agenda” embedded in these assessments. The Central Government last year announced plans to devise new measures of socio-economic progress, discarding “misleading” international parameters that measure childhood stunting, female labour force participation rate and life expectancy at birth.
The grouse with democracy assessments is that the methodology is flawed, sample sizes inadequate, and that these indices favour cultural bias and subjective opinion over objective metrics. India, for instance, ranks between Niger (which is ruled by a military junta) and Ivory Coast, and is in the same category as Palestine. Any yardstick of democracy — be it fair elections or electoral participation — would suggest India is “doing as well as any other democracy,” Mr. Jaishankar said.
Little is known about India’s proposed index. The think tank Observer Research Foundation (ORF, partly funded by Mukesh Ambani-owned Reliance Foundation) will be the brains behind the operation; the methodology has been “peer-reviewed” by experts and is expected to be announced soon. An attempt at an indigenous democratic rating system was made once in 2021, according to a Hindustan Times report, when India was mulling a “world democracy report” and a “global press freedom index,” after being downgraded in V-Dem Institute and Freedom House’s reports.
How many global democracy indices are there?
Many. The V-Dem Institute at Sweden’s University of Gothenburg has published its annual assessment of 202 countries since 1789. Since 1973, the U.S.-based Freedom House has come up with its assessment of civil liberties and political rights. The Economic Intelligence Unit assesses the electoral, liberal, participatory and effective nature of democracies since 2006. Others include: the Lexical Index, Boix-Miller-Rosato coding, Bertelsmann Transformation Index, Worldwide Governance Indicators, and International IDEA’s Global State of Democracies report.
There are four broad types of data that these indices use:
- Observational data (OD) — data on observable facts, such as voter turnout rates
- ‘In-house’ coding, where researchers assess country-specific information using academic material, newspapers, etc.
- Expert surveys, where selected experts from a country provide a subjective evaluation
- Representative surveys, where a selected group of citizens offer judgments
There are many approaches to measuring democracy, some using facts, some judgment and some a mix of both. The Lexical Index, for instance, relies on observational data in addition to evaluation by in-house researchers based on academic research. V-Dem, on the other hand, uses aggregate expert judgments from a pool of 3,500 scholars and almost 30 million data points. The model “algorithmically estimates both the degree to which an expert is reliable relative to other experts, and the degree to which their perception differs from other experts to come up with the most accurate values for every parameter,” The Hindu previously explained.
The question of the dataset used is controversial — each approach has strengths and setbacks. Scholar Svend-Erik Skaaning in a paper identified the main priority as establishing a high degree of concept-measure consistency — “the extent to which the indicators capture all of the components of the core concept of interest (and only those), and the extent to which they do so in a precise and unbiased manner.”
”...no type of data is superior to the others in all respects,” he added.
Also read: Why 2024 could be the ‘make-or-break’ year for democracy | Explained
The Office of the United Nations High Commissioner for Human Rights has endorsed the use of observational, objective data over judgment-based methodology for such assessments to make them “more broadly acceptable.” Others, however, find fact-based metrics “insufficient”, and expert intervention necessary, to capture on-ground realities of governance. In an Article-14 interview, V-Dem’s director Staffan Lindberg mentioned expert surveys are “not looking for an opinion” but “for knowledge.”
How is democracy defined?
Each index asks and responds to a different question evaluating the health of democracy. All agree that democracy is a political system in which citizens get to participate in free and fair elections (electoral democracy). Democracies are also liberal societies, which invest in the civic rights of citizens and offer them protections. Indices like V-Dem’s, Economist Intelligence Unit and the Bertelsmann Transformation Index assess other dimensions as well: Is the democracy ‘participatory’, are citizen groups and civil society organisations functional? Are decisions made deliberately, in the best interest of all people, rather than through coercion or minority group interests? Is it egalitarian — are economic and social resources distributed equally?
The approach varies vastly too. Some use only two indicators while others have more than 400; the weightage assigned and aggregation model followed also fluctuates across projects. V-Dem’s researchers code a series of indicators for 12 areas across media, civil society, political parties, judiciary, and civil liberties; each area is assigned five experts. EIU relies on regular citizen surveys to rank 60 indicators. Together, these indices “comprise a combination of quantitative assessments like the distribution of seats in the national legislature among political parties, and qualitative judgements like whether safeguards against corruption are effective,” a study by University of Pennsylvania researchers noted.
Some rate democracies on a spectrum. The EIU rank goes from 0 to 10; V-Dem’s goes from 0 to 1. Others also classify the degree of democracy— say, from an autocracy to anocracy to democracy in Polity’s Index; or ‘not free’ to ‘electoral democracy’ in Freedom House’s ranking (India is a “partially free democracy”). The V-Dem Institute sorts countries into four regime types: Liberal Democracy, Electoral Democracy, Electoral Autocracy, and Closed Autocracy. India since 2018 is classified as an “electoral autocracy” due to a visible “democratic backsliding”,“crackdowns” on civil liberties, and Prime Minister Narendra Modi’s policies that have “fomented anti-Muslim feeling and religious strife and damaged the political fabric of the country.”
What are the limitations of global indices?
The first and the most frequently cited criticism: there is a degree of subjectivity that tugs at the indices’ credibility and precision. Regardless of the scholarly pool and aggregation model used, evaluations are still based on the judgement of researchers and coders, rather than discernible, tangible characteristics. V-Dem’s “egalitarian” indicator, for instance, assesses the equality of social groups in the political arena — an equivocal question in comparison to say, how many political parties are present in the country. In a 2022 interview, Salvatore Babones, an associate professor at the University of Sydney, questioned the strength of the data, evidence and neutrality of people (professors, authors, and intellectuals) who conducted the surveys. “We don’t have access to the list of experts they rely on...it is evident that Indian intellectuals and university professors have a clear bias against Narendra Modi and BJP,” he said.
A project investigated the degree of expert biases in some indices and found them to be limited. Scholar Paul Staniland, who studied V-Dem’s assessment of India since 1947, concurred, telling the BBC in 2021 that “there’s not an obvious anti-right-wing bias.” While India’s ranking was lower during the Emergency in the mid-Seventies, India did not slip in rankings between 1998 and 2004 when the BJP was in power.
Mr. Lindberg said research ethics and European and Swedish laws restrict the project from disclosing researchers’ identity. They also have between 30 to 40 experts for India, academics who have verified scientific knowledge and “know the most and have published scientific articles on elections in India or the judiciary in India,” he said.
Some indices have a “differential item functioning” to adjust for variations and biases between experts. V-Dem’s model requires researchers to state uncertainties, compare disagreements between experts, and produce best and worst estimates of many indicators. A paper found this methodology “better capture[d] the opaqueness of contemporary autocracies.” For instance, it classified regimes with electoral manipulation or infringements of political freedoms as “electoral autocracies.”
There is also a concern over the workings of the index’s aggregation model: experts’ judgment is used to decide which metrics to include and how to weigh each appropriately. Bastian Herre in Our World in Data noted it remains unclear why some subindices were chosen; and “why two subindices, elected officials and voting rights, are weighted less than the others.”
Another concern is over the scope of countries included in these indices. Only some — such as Freedom House and Lexical Index —- survey non-independent and microstates. Smaller countries may thus be overlooked in certain cases.
The next criticism is of a perceived ideological discrepancy, partly due to the amorphous definition of democracy itself. Lesotho, which suffered a military coup in 2014, is assigned a higher score than India. And if economically unequal countries such as Brazil are democratic, how is India classified as an electoral autocracy? India’s Foreign Ministry in 2018 called the V-Dem report declaring India an electoral autocracy “inaccurate and distorted.”
What are democracy indices good for?
Scholars, including Mr. Staniland, agreed the indices “capture important big-picture dynamics and trends” in democracy. They offer ways to benchmark the strengths and weaknesses of political regimes, and make different components comparable over time periods and across geographies.
There is no consensus on how a democracy should be defined, how to quantify these components, or how to combine them in a single index of democracy. There is no singular, perfect democracy index, just like there is no singular definition of democracy. Mr. Herre suggested using these assessments as different “tools” to employ in interrogating different questions about the health of a democracy. For example, Freedom House may be the right place to look if assessing the political and civic freedom in a democracy. If looking at different dimensions — electoral, liberal, participatory, deliberative, effective — the BTI may hold some answers. For big differences observed by experts in political regimes in the last 200 years, V-Dem’s Regimes of the World dataset can help.
- Indices like V-Dem’s, Economist Intelligence Unit and the Bertelsmann Transformation Index assess other dimensions as well: Is the democracy ‘participatory’, are citizen groups and civil society organisations functional? Are decisions made deliberately, in the best interest of all people, rather than through coercion or minority group interests? Is it egalitarian — are economic and social resources distributed equally?
- The first and the most frequently cited criticism: there is a degree of subjectivity that tugs at the indices’ credibility and precision. Regardless of the scholarly pool and aggregation model used, evaluations are still based on the judgement of researchers and coders, rather than discernible, tangible characteristics.
- Scholars, including Mr. Staniland, agreed the indices “capture important big-picture dynamics and trends” in democracy. They offer ways to benchmark the strengths and weaknesses of political regimes, and make different components comparable over time periods and across geographies.
