The NCAER’s new dataset has opened a window on to parts of the Indian story that we’ve known little about for nearly a decade.

For the last two weeks, The Hindu has reported exclusively on an exciting new dataset, the National Council for Applied Economic Research’s (NCAER) India Human Development Survey (IHDS) 2011-12. It’s an exciting dataset for three big reasons.

One, it’s a 42,000 household-strong nationally representative survey with questions on both economic and social issues. Two, it has substantial panel data – which means data that allows for comparison over time – because the NCAER researchers went back and interviewed 80% of the households they spoke to for the first round of the IHDS, in 2004-5. Three, it is a credible and independent body giving researchers, the media and readers access to information that we otherwise depend almost entirely on the government for.

The NCAER gave The Hindu access to the data ahead of its publication. Their researchers wrote five opinion pieces for us looking at five major transformations in the country: the status of women, whether the Mahatma Gandhi National Rural Guarantee Scheme is doing its job, the revival in the Public Distribution System despite its flaws, auditing the performance of public education and healthcare, and on what is happening with income growth and inequality.

Using the NCAER data (but reporting independently on it), I wrote five news stories for the paper:

1. The first showed that an alarming number of women have little say in the fixing of their marriage, and other constrained freedoms. There was also an infographic with it that illustrated the data.

2. The second showed that a combination of farm and non-farm work is now, for the first time, the most common form of employment in rural areas (as against agriculture alone)

3. The third showed that the amount children learn in school is mediated by caste, class and location – and is falling, all round.

4. The fourth showed that there has indeed been a big increase in incomes, while access to services has creeped along. This is of note particularly because this is India’s only source of income data – the NSSO measures consumption only – thereby revealing India’s first credible income data in seven years. An accompanying infographic showed what class differences mean in concrete terms.

5. The final part forthcoming gave a more realistic picture of what the Indian middle class really looks like.

For a data journalist, it’s been a real joy having such high quality data to work with and play with for the last few weeks. We hope you’ve read along and found numbers that shone a light on to something you didn’t know.