The importance of good data

In a democracy, it is important for government agencies to disseminate data in a timely manner

Concept of business movement, Arrows

After a delayed start, the Southwest Monsoon recently wreaked havoc in Kerala, Maharashtra, Karnataka and other parts of India. In such a situation, a data journalist would have ideally looked for weekly comparisons of rainfall data across the affected States to determine when the rains began to intensify; historical data to determine how the rainfall compared to the five- or 10-year average; and whether there was a ‘tipping point’ in the amount of rainfall.

However, getting such data is difficult in India as the India Meteorological Department (IMD) has only two options on its website: the current week’s rainfall numbers and the cumulative rainfall numbers of the season. To be fair, IMD also provides district-wise and State-wise data, but for the same two options. And while five-year district-wise data are available, each district has to be selected separately to access the data, a cumbersome and time-consuming process.

Outdated data

It is not just the lack of data which is a problem but also the fact that a lot of it is outdated. The Central government’s initiative to have an open data portal is appreciable, but the datasets are rarely updated. Monthly data on rainfall are available for each sub-division, starting from 1902, but they stop with 2017. Similarly, the Census 2011 data on Indian migration, released last month, was well organised and provided insights into why people migrate, which States they come from and go to, etc. However, nine years have passed since the data was collected. Cities have undergone massive changes since then causing changes in patterns of migration too.

Furthermore, there are several datasets which are redundant. For instance, while there are data available from 1902 to 2017 for rainfall, the datasets for 1902-2015 and 1902-2016 are unnecessary.

Of course, collating the details of 1.21 billion people based on different parameters is a monumental task. Extremely granular data may not be required to give a snapshot of how things stand. But what is required is timely dissemination of good-quality data, whether on employment (delayed this year) or on crimes (the National Crime Research Bureau is yet to release data from 2017).

Problem of ‘dashboardisation’

Instead of releasing data in a more timely manner, the government seems more keen on ‘dashboardisation’. There is a perceptible shift from the timely release of structured data to publishing data dashboards with cumulative statistics. There are now dashboards for everything — from the number of BHIM transactions to the number of ‘Smart City’ projects taken up. The problem with this is that we never get the full picture. For instance, during Question Hour in Parliament, if questions are asked on start-ups, statistics are provided only about the start-ups under the Centre’s Startup India Scheme. Similarly, questions on employment generated by MSMEs are answered only with data on employment generated under the Prime Minister’s Employment Generation Programme.

Why is the full picture so important? Let’s take the case of a recent report that said that in an Uttarakhand village called Uttarkashi, not a single girl child was born in 133 villages in three months. A closer look revealed that while concerns about foeticide and infanticide were valid, another vital piece of data in this story would have assuaged concerns — the fact that in another 129 villages in the same region, no male children were born in the same period.

Merely putting out some numbers on a dashboard do not help data journalists, researchers and the public study data over time and identify trends. It is important to know not only the numbers but also know how these numbers were arrived at. This is not the case right now.

Data are crucial, particularly for journalists, so that they can report accurately on various issues, especially at a time when fake news is rampant. It is a fundamental right for the public in a democracy to be informed. The government can earn the trust of the public only when it is transparent about numbers and timely in their dissemination.

varunkrishnan.b@thehindu.co.in