Over the past two months, Indian national statistics and the organisations that administer them have faced a volley of criticism. In January, two independent members of the National Statistical Commission resigned in protest , over alleged suppression of economic data by the government. More recently, amidst growing scepticism regarding India’s official statistics, more than a hundred scholars comprising economists and social scientists released a statement decrying the fall in standards of institutional independence, suggesting political interference as the cause. Kaushik Basu, a former chief economist of the World Bank, also recently bemoaned the declining credibility of India’s official statistics .
Pioneering history
While declining data quality has been an issue for a while, concern over institutional independence is new. What several of these criticisms reference is the fact that India’s national statistics were once internationally renowned among economists and policy professionals for their reliability. In the decades following World War II, India had reason to be proud not only of the institutional independence of national statistical bodies but also — uniquely among developing countries — of a pioneering history of independent data collection and publication. But what exactly was that history?
ADVERTISEMENT
The growth of India’s vast national statistical infrastructure dates back to its first decade as an independent country. The birth of a new nation led to an explosion of national statistics, based on the need to plan the economy through Five Year Plans. These years would see the establishment of the office of the Statistical Adviser to the Government, bi-annual National Sample Surveys (NSS), the Central Statistical Organisation (CSO), and National Income Committees (that made the estimates similar to GDP measurements). The moving spirit behind these developments was Prasanta Chandra Mahalanobis, whom Jawaharlal Nehru described as the “presiding genius of statistics in India,” and the institute that he had founded in Calcutta in 1931, the Indian Statistical Institute (ISI).
While the British colonial government had made efforts to collect statistics on the subcontinent from the early 19th century, these were provincially organised and geared towards trade and administration. On the eve of World War II, it had become apparent, both to the colonial government and the Indian National Congress, that any concerted postwar developmental effort would require fine-grained statistical information on the national economy. Nehru, Chairman of the Congress’s National Planning Committee, called attention in 1938 to the “fact of the absence of accurate data and statistics.”. Even a decade later, he would admit, “we have no data,” as a result of which, “we function largely in the dark.”
It was this need that would elevate the profile of the Indian Statistical Institute and Mahalanobis, both internationally feted in the 1940s for their scholarly contributions to theoretical and applied statistics. ‘The Professor’, as Mahalanobis was known to associates, was involved in the discussions that led to establishment of the UN Statistical Commission in New York (a body that he would be voted Chairperson of several times during the 1950s). As a pioneer in the emerging field of large-scale sample surveys, he would also be the force behind creating the UN Sub-Commission on Statistical Sampling in 1947, co-authoring the textbook on the subject in 1950.
ADVERTISEMENT
Launching sample surveys
By the middle of the twentieth century, the Indian Statistical Institute was globally recognised as a leader in the field of sample surveys. It would soon even begin training statisticians from other developing countries. The famed English statistician R.A. Fischer observed that its achievements “brought India not far from the centre of statistical map of the world.” The Institute’s fingerprints were readily apparent in the creation of India’s National Income Committee, the Central Statistical Organisation, the International Statistical Education Centre in Calcutta, and the National Sample Survey — all created around the mid-century mark.
The inaugural National Sample Survey was, as the Hindustan Times reported in 1953, “the biggest and most comprehensive sampling inquiry ever undertaken in any country in the world.” These were, as Nobel Prize-winning economist Angus Deaton put it, the “world’s first system of household surveys to apply the principles of random sampling.” The sheer scale seemed foolhardy, even to sympathetic statisticians. As the American statistician W. Edwards Deming recalled: “We in this country [U.S.], though accustomed to large scale sample surveys, were aghast at Mahalanobis’ plans for the national sample surveys of India. Their complexity and scope seemed beyond the bounds of possibility.” The first survey, performed by hundreds of dedicated staff, involved manifold challenges according to reports: in Odisha’s forested areas investigators had to be accompanied by armed guards; in the Himalayas they waited for the snows to melt in the passes; in Assam they encountered “naked tribes” who did “not know what money means”; and elsewhere they waded through “deep jungles infested with wild-beasts and man eaters.”
High-definition snapshots
The results of the National Sample Survey offered high-definition snapshots of the country’s material life — casting light on cost of living, crop estimates, household consumption, industry, trade, and land holding patterns. Twenty years later, the once sceptical Edwards Deming was now a convert: “No country, developed, under-developed or over-developed, has such a wealth of information about its people as India.” The contemporary Singaporean statistician Y.P. Seng observed that by comparison that China had “no genuine statistics” and so India’s example of using surveys would “serve as a guide and an example worthy of imitating”.
The Planning Commission, beginning in 1962, used the data the National Sample Survey generated by its household surveys to craft the country’s poverty line. India was a frontrunner in this regard: the United States developed its own poverty line three years later. With their combined influence on the UN Statistical Commission and the UN Sub-Commission on Statistical Sampling, the Indian Statistical Institute and the National Sample Survey continue to have a lasting impact on estimating poverty across the developing world. Methods pioneered by the National Sample Survey have become the norm for household surveys across the globe. For example, the Living Standard Measurement Study surveys conducted in several countries by the World Bank can trace their lineage back to the work of Indian statisticians associated with the Indian Statistical Institute and the National Sample Survey.
An anomaly?
This distinguished history, which India can claim with pride, makes the recent undermining of the credibility of our statistical output especially regrettable. We can, however, ensure that when we look back on this several years from now, it represents an anomaly rather than a lasting, irreparable loss of institutional credibility.
Nikhil Menon is an Assistant Professor of History at the University of Notre Dame, U.S. He is writing a book on economic planning and democratic state building in independent India