This is not what the doctor ordered

Evidence-based medicine (EBM) is the new mantra. It evaluates the strength of the evidence of benefits and risks of treatments. Its processes mandate a systematic search of international medical literature to ferret out relevant research. It attempts to assess the quality of the evidence and ranks its sources. Its rigorous methodology, elaborate checklists, detailed assessments and quality controls have raised research standards. It recommends that the best available evidence be used to aid clinical decision-making and policy. It has improved the evidence base. EBM has changed medical practice.

Medical journals now have regular EBM updates. Traditional summarisation of evidence has given way to systematic reviews. Medical research now needs to comply with EBM guidelines and satisfy its criteria. Nevertheless, its widespread acceptance and use also raise many questions.

Evidence is not generalisable: Randomised controlled trials (RCT) are the cornerstone of EBM. The results of a single RCT or a systematic review of many trials provide evidence about the efficacy of treatments (i.e. “The treatment works somewhere”). However, it does not necessarily provide evidence of effectiveness in clinical practice (i.e. “The treatment works widely”). Extrapolating knowledge gained from RCTs to individual patients across different settings is problematic. Such extrapolation requires much more evidence, which is often unavailable.

Evidence for efficacy (“Can it work?”), effectiveness (“Does it work in practice?”) and efficiency (“Is it worth it?”) all need to be considered. Most trials test for efficacy in ideal situations using detailed protocols, carefully selected patients and placebo controls, with good treatment compliance and intensive follow-up. These ideals are rarely achievable in routine clinical practice with its poor diagnostic accuracy, patient compliance and coverage. Studies, which examine effectiveness, are few and often document results that are different from the original seminal investigations.

Hawthorne effect: There are many examples of efficacy trials comparing specific interventions with “usual care,” which have shown superiority of the interventions concerned. However, much of the success reported may be due to the Hawthorne effect, where the care received by patients in the experimental arm exceeds that received in the “treatment-as-usual” arm. The motivational response seen may be due to the interest, care and attention received (both through observation and assessment), rather than to the specific intervention. The inability to implement mental health programmes in low and middle-income countries, despite the reported success of model programmes and RCTs, also argues for such an effect, in addition to operational difficulties in the real world.

Evidence fades: First generation anti-psychotic medications were effective in treating schizophrenia. They were the established standard of care during the second half of the twentieth century. Subsequently, several large and well-designed studies in the 1990s demonstrated the efficacy of new compounds in the treatment of the disease. These new drugs soon became the gold standard. Their widespread prescription and use produced fortunes for their manufacturers.

A decade after the new drugs were established as benchmarks, reports to the contrary started to appear. Large and well-conducted trials showed that the older low-cost anti-psychotics performed as well as the newer medication. Yet, the use of the newer and more expensive medication continues unabated. Similarly, the use of anti-depressant medication as a panacea for all types of depression has been challenged recently. New guidelines do not recommend pharmacological interventions for mild and moderate depression.

Cost of generating evidence: EBM has unintended effects. It is now not possible to generate good evidence without substantial financial support. Competitive funding for research makes it difficult to obtain grants for replicating studies in different contexts, thus making effectiveness trials rare. Pharmaceutical companies focus on efficacy studies in ideal conditions. Independently funded trials are uncommon. There is evidence to suggest that industry-sponsored trials are more likely to suggest that new treatments are superior than independently funded trials. Much of the research, which has demonstrated reduced effectiveness of the so-called new standard treatments, was carried out by independent agencies.

The shortage of new medications in the drug development pipeline has not prevented the pharmaceutical industry from making minor modifications in existing molecules and demonstrating the efficacy of these, comparable with the original drugs. The variants are introduced as patented new treatments and are priced much higher. Efficacy studies, without concomitant investigations related to effectiveness and efficiency, are used to sell the variants as the latest advance.

Can we trust the evidence? Biases, discrepancies, conflicts of interest and their impact on the evidence have been highlighted in medical literature. Weakness in design, protocol violations and high rates of dropouts in clinical trials and ghostwriting by pharmaceuticals on behalf of academics are major problems. Cherry-picking data to publish positive results while suppressing uncomfortable truths is not uncommon. While attempts at regulation and registration of trials have made an impact, more needs to be done.

The Indian context

India, with its expanding middle-class, flourishing private health sector and its numerous trained health-care professionals, is a goldmine for the pharmaceutical industry. Pharmaceutical companies spend small fortunes on advertising and selling their new products, without investing in studies to prove their effectiveness in the Indian context. Indian medical establishments and leaders are willing to do efficacy studies for western funders. The knowledge that effectiveness trials in the diverse and difficult Indian environment are not a viable financial option and that the results obtained may not be generalisable to the average Indian context is of no consequence. Differences in patients, conditions, contexts and settings are dismissed as trivial. Cost-effectiveness and efficiency studies are rare.

On the other hand, measures such as distribution of free medication samples for physicians' use and sponsored seminars in exotic locations are commonly employed to introduce new products to physicians. The indiscriminate use of newer antibiotics is an example of a situation with assured profits for pharmaceuticals, physicians and pharmacists at the cost of the individual and nation's health. Physicians are under pressure to keep up with international medical literature and prescribe the latest medication, which has proven efficacious in the West but with no evidence of effectiveness and efficiency in the Indian context.

The debate

Many issues have been highlighted in literature. Should we prioritise effectiveness trials of new interventions before licensing them for clinical use? Should we mandate the need for efficiency trials in India before we accept the most recent but exorbitantly priced interventions? Are results of statistically significant early studies to be considered outliers, which take years to reveal their true nature? Why do we continue to believe in initial dramatic results, despite the demonstration of the limited impact of these interventions?

While rigorous assessment of evidence is crucial to medicine, evidence can also be an instrument for institutional control and policy. EBM is now used to oversee the practice of medicine, altering the balance of power among physicians, payers, providers and patients. EBM reduces the discretion and autonomy of physicians and helps promote the shift in power. It facilitates the work of medical managers and increases the control by the pharmaceutical and health industry, third-party payers and patients. While in the past the authority of doctors prevented questioning of their clinical choices, with EBM, physicians will have to justify their decisions, thereby reducing the clinical discretion of doctors. Clinical insights, experience and contextual understanding have given way to data-driven efficacy trials with little regard for effectiveness and efficiency. The need for huge monetary investments to generate EBM data necessarily implies that those with financial influence will control research and practice.

The EBM movement hoped that better evidence would lead to improved decision-making, medical practice and health policy. However, the technical approaches advocated by EBM are not necessarily value free or above specific interests. Medicine and health care, to paraphrase German pathologist and statesman Rudolf Virchow, is politics on a grand scale. The health sector is a powerful player in national economies. The technical language of EBM can hide major political disputes and conflicts of interest. We should be aware that technical fixes do not necessarily avoid political conflicts related to income, turf and goals of medicine and policy. The politics of expertise and evidence has become more complicated.

Clinical practice is multifaceted and complex. The context involved and clinical, psychological, social and economic issues need to be considered. Co-existent medical conditions and risks, drug interactions and patient perspectives also impact management.

Evidence-based medicine has improved clinical management. However, the uncritical use of the recommended technical fixes hides many issues. Financial interests and sponsorship trump EBM guidelines; efficacy in ideal conditions trounces the need for effectiveness and efficiency evaluations in context of practice. The contemporary confusion calls for a nuanced understanding of issues involved and an enlightened approach to health care.

(Professor K.S. Jacob is on the faculty of the Christian Medical College, Vellore. This article is based on a recent editorial published by him in the National Medical Journal of India.)