A blog that explores happenings in the realm of data and provides insights into the world we live in. Ultimately, people matter, not the numbers.
December 13, 2014 Rukmini S
Comment   ·   print

How the type of question asked affects survey findings

If you’ve always suspected that surveys are highly dependent on the way the question is asked, here’s some empirical proof that you’re right. The way questions about sanitation are asked has significantly affected data on the extent of open defecation in India, new research reveals.

In a working draft paper, economists Diane Coffey and Dean Spears of the Research Institute for Compassionate Economics compared seven recent studies on toilet use in rural India. They found that surveys which asked a balanced question about open defecation or latrine use, individually, for each person in the household were able to document more open defecation among those who owned toilets than unbalanced survey questions that grouped household members.

So who interviews each person in the household? Three important national surveys do not - The 2011 Census of India, the India Human Development Survey (IHDS), and the National Family Health Survey (NFHS), all report a single household-level measure that combines behaviour and ownership. Four studies considered by Ms. Coffey and Mr. Spears partially disaggregate members of the household. Three others fully disaggregate, i.e. they ask every member of the household.

What is a balanced question? One, say the researchers, in which multiple options are presented without evidence of a bias in favour of one of the options. “Do you use a toilet or do you defecate in the open?” would be a balanced question in this case, as opposed to “Do you use a toilet?” “When asking unbalanced questions, it is easier for surveyors and respondents to speed through yes/no questions, or to tick off default responses, rather than noticing the verbal and non-verbal cues of respondents, which leads to the recording of more accurate information,” they say.

This difference in open defecation rates measured by the two types of surveys is statistically significant (meaning very unlikely to be due to chance alone) and is not because of differences in overall open defecation rates across study sites.

These findings are, for me, particularly significant because the National Sample Survey Office, India’s official source of statistics, does not ask either a balanced question or ask each member of the household about toilet use. This sadly means that in its most recent round (2011-12), India likely underestimated the extent of open defecation in India. Of the seven studies Ms. Coffey and Mr. Spears looked at, the NSSO in fact found the least open defecation among people who have toilets.

December 6, 2014 RUKMINI S.
Comment   ·   print
Whose views does the Corruption Perception Index reflect? »
November 19, 2014 RUKMINI S.
Comment   ·   print
New numbers shed light on what the typical Indian marriage looks like »
November 18, 2014 T. Ramachandran
Comment   ·   print
The focus on World Toilet Day, which falls on November 19, is squarely on a large section of the global population that does not have access to toilets. Particularly women, for whom access to toilets could mean better security. »
November 7, 2014 Rukmini S.
Comment   ·   print
With the Swachh Bharat campaign, is India focussing its attention on the right things? »
Pages: «  ‹    ›  »
July 14, 2014

Movement to Iraq increased during 2013-14; there has been an overall increase among those going abroad with 'emigration check required' (ECR) clearance between 2011-12 and 2013-14. There was a declining trend in the case of some countries, though. »

July 10, 2014

Regional differences and meat-eating in India »

July 7, 2014

Do statistics tell us that the dowry law was being misused? »

May 26, 2014

This unusual looking map seeks to depict the new electoral arithmetic in which the BJP's dominance is overwhelming. »

May 20, 2014

What first-past-the-post means for India, how proportional representation would have looked, and how the choice is not simple. »

Pages: «  ‹    ›  »



Please remember to logout of facebook after you complete this session.