Online edition of India's National Newspaper
Saturday, Aug 06, 2005

News: Front Page | National | Tamil Nadu | Andhra Pradesh | Karnataka | Kerala | New Delhi | Other States | International | Opinion | Business | Sport | Miscellaneous | Engagements |
Classifieds | Employment | Obituary |

Opinion - News Analysis Printer Friendly Page   Send this Article to a Friend

Don't trust the headlines

John Allen Paulos

Medical research may make great headlines in the media, but new analysis shows too many studies later prove to be less than accurate.

HOW MANY times have you heard people exclaim something like, "First they tell us this is good or bad for us, and then they tell us just the opposite"? In case you need more confirmation of the iffiness of many health studies, John Ioannidis of the University of Ioannina in Greece, writing in the Journal of the American Medical Association, recently analysed 45 well-publicised studies from leading journals appearing between 1990 and 2003. His conclusion: the results of approximately one third of these studies were flatly contradicted or significantly weakened by later work.

There is the well-known story of hormone replacement therapy, which was supposed to protect against heart disease and other maladies, but apparently does not. A good part of the apparent effect may have been the result of attributing the well-being of upper middle class health-conscious women to the hormones.

Another bit of health folklore "everybody knows" that turned out to be unfounded is vitamin E's protective effect against cardiac problems. Not so, says a recent large study. And how about red wine, tea, fruits, and vegetables? Surely the anti-oxidant effect of these wondrous nutrients cannot be doubted. Even here the effect appears to be more modest than Pinot Noir lovers, among others, had thought.

A common procedure to remove fat from neck arteries, prescription drugs used by millions of people — the examples go on and on, but the general point is that a single health study by itself cannot be taken as indubitable. The totality of the available evidence, appropriately weighted, is what counts, and this balanced appraisal is difficult to fit into a news article, much less into a catchy headline.

One obvious problem is that studies vary in size and quality. Some are well-designed, others not, yet most media reports give all of them the same status — the medical variant of "astronomers say one thing, astrologers another, so let's hear from both." Margins of error, low correlations, or very large ones that mask confounding variables seldom make it into the league of news stories, whereas "X will cure you" or "Y will kill you" always seem to.

Many health studies rely on self-reporting, which is notoriously unreliable. The number of heterosexual sex partners reported by men, for example, is almost always considerably larger than the number reported by women. And the evaluation of all studies must contend with wishful thinking: people naturally want to believe in the value of new treatments, sometimes so much that their critical faculties are dulled or extinguished altogether. In the other direction, people often over-react to bad news and fall victims subject to the "tyranny of the anecdote." For example, TV viewers see parents keening about the unfortunate effect of some vaccine on their child and give little weight to the hundreds of thousands of children who have benefited from the same vaccine. A distinction from statistics is marginally relevant. We are said to commit a Type 1 error when we reject a truth, and a Type 2 error when we accept a falsehood. In listening to news reports, people often have an inclination to suspend their initial disbelief in order to be cheered and thereby risk making a Type 2 error. In evaluating medical claims, however, researchers generally have an opposite inclination to suspend their initial belief in order not to be beguiled and thereby risk making a Type 1 error. There is, of course, no way to always avoid both types of error, and we have different error thresholds in different endeavours.

So what should you conclude about, say, a new study that flavonoids in dark chocolate help lower blood pressure? It is your call, but how credible you find this chocolate study may say more about your psychology than the biochemistry of chocolate.

As I have written before (although with a different number), it has been conclusively established that 43.58871563 per cent of all statistics are made up on the spot. — © Guardian Newspapers Limited 2004

(John Allen Paulos is a professor of mathematics at Temple University, Philadelphia.)

Printer friendly page  
Send this article to Friends by E-Mail


News: Front Page | National | Tamil Nadu | Andhra Pradesh | Karnataka | Kerala | New Delhi | Other States | International | Opinion | Business | Sport | Miscellaneous | Engagements |
Classifieds | Employment | Obituary | Updates: Breaking News |

News Update

The Hindu Group: Home | About Us | Copyright | Archives | Contacts | Subscription
Group Sites: The Hindu | Business Line | The Sportstar | Frontline | The Hindu eBooks | The Hindu Images | Home |

Copyright 2005, The Hindu. Republication or redissemination of the contents of this screen are expressly prohibited without the written consent of The Hindu