INTRODUCTION AND OVERVIEW

This chapter covers some of the generic issues related to evidence-based medicine (EBM), with particular emphasis on how these apply to integrative general practice and complementary therapies. Although EBM principles are a vital part of the armamentarium of the modern doctor, they are a two-edged sword.

MEDICAL OPINION VERSUS EVIDENCE

Traditionally, doctors in training have looked to ‘expert opinion’ from senior medical staff as their primary source of guidance on clinical decision-making. Their long experience and learning from prior generations of medical experts was deemed sufficient authority and was rarely questioned. Then the general practitioner (GP) would also add their personal clinical experience as it was gathered over the years to formulate and adapt their particular approach to clinical problems.

As simple and intuitive as this approach is, it has always had deficiencies. Our perceptions, attitudes and range of options can be informed by expert opinion, but equally they can be limited or biased by them. A recent study concluded that:

Put another way, when we are confident of our opinions, and have had them handed down to us from respected authorities without question, and when we are not questioned about the validity of our opinions, we are less likely to determine whether they are based on evidence.

POTENTIAL AND LIMITATIONS OF EBM

As history is written, the exponential growth in EBM gained momentum in the late twentieth century. Published data increasingly revealed:

The Cochrane Collaboration was formed out of this movement, and its systematic reviews follow a transparent process ‘intended to minimise bias’. We accept this as almost a motherhood statement, but the processes inherent in EBM contain a number of major flaws.

In recent decades, EBM has rightly assumed a leading role in informing medical practice. Stratifying and analysing the results of all published trials for a disease or an intervention provides some guidance on the three pillars—quality, safety and efficacy—and allows for comparisons to be made. To make clinical decisions and formulate clinical guidelines that are contrary to evidence is at best ineffectual and wasteful of resources, and at worst harmful to the patient.

However, a bland statement that the clinician should make decisions that are evidence-based is not as simple as it seems, for a number of reasons.

Some particular limitations of EBM in the general practice setting are summarised by Stephenson:²

It is for these reasons that the best approach to clinical decision-making will combine knowledge of the best available evidence with:

Many complementary therapies are particularly affected by the points raised above. Naturally occurring products, for example, are not patentable, and so they don’t attract the large research budgets that patentable pharmaceuticals do. Even where there is mounting evidence that a particular complementary therapy is safe and effective, it is often treated with scepticism by the medical mainstream because it falls outside the imaginary boundary of conventional medical practice. As clinicians we need to be interested in what is safe, works and is cost-effective regardless of which side of an artificially created demarcation line between complementary and conventional therapies it comes from.

LEVELS OF EVIDENCE

EBM experts tend to refer to the ‘level of evidence’ for or against a particular therapy or intervention. Broadly speaking there are four main levels of evidence, although there are subdivisions within these:

Obviously, level 1 evidence is considered the highest; the relative weight given to each of the others diminishes as you go down the levels. In reality, however, it is not as simple as this. For various reasons, some interventions or issues do not lend themselves to RCTs or cannot attract the necessary funding for high-quality RCTs and will therefore never have level 1 evidence supporting them. This does not mean that they are invalid, but simply that one needs to be flexible in interpreting evidence and that EBM needs to be supported by common sense and clinical experience.

Meta-analyses also rely on the publication of all available studies. Because publication bias is more likely to exclude negative findings, meta-analyses of published studies tend to skew to positive findings.

EBM reviews sometimes provide challenging findings. For example, a meta-analysis published in The Lancet gave surprising results on the efficacy of homeopathy, finding that the odds ratio for positive trial results was 2.45 in favour of homeopathy.³ Though more high-quality RCTs are required, either homeopathy is clinically effective for a range of conditions or there is a publication bias in favour of homeopathy. Thus, even with ‘level 1 evidence’ a clinician still needs to interpret the findings and decide what to make of them.

Patients and the general public are often not aware of levels of evidence, and nor can they easily interpret evidence for themselves. Furthermore, they can be significantly affected by misleading marketing. The GP therefore has a vitally important role in helping people to:

The evidence that most patients regard first and foremost is the evidence from their own personal experience or the experience of people close to them. The question most relevant to them is, ‘Does it work for me?’ A doctor telling a patient that a therapy, such as a herbal treatment, is not supported by evidence when it is clearly providing benefit for them will not convince them and could alienate them from the doctor. It may be more useful to say that it has been little researched but that the person’s experience is also a valid form of evidence. It is also unconvincing for the patient to be told that a therapy is well supported by evidence when it is not working or they are experiencing side effects that are worse than the condition for which they are being treated. Thus the findings and recommendations arising from EBM need to be interpreted and communicated with sensitivity and in a way that is appropriate to the individual patient.

CAM RESEARCH

For many complementary therapies it is difficult to attract research dollars. From a marketing perspective, products that are not patentable are a less desirable topic of investigation. Independent research funds are therefore often the only avenue available for CAM researchers. Without such research data, clinicians and patients are making less-informed decisions. In Australia, independent funds for CAM research were provided for the first time in 2008. With the establishment of the National Center for Complementary and Alternative Medicine (NCCAM) in the USA there was a rapid rise in research funds and competitive grants won for CAM research.

By and large, there are too few studies of most complementary therapies, and those that are done are often of poor quality, due to limited funds or inexperienced researchers. Nevertheless, the number of CAM citations in the medical literature has grown exponentially in recent years.

EVIDENCE AND COMPLEX INTERVENTIONS

Another problem with CAM research is that many therapies are practised ‘holistically’, with the therapy being individualised to the patient, and some therapies—such as acupuncture, lifestyle interventions and counselling—present difficulties with placebo trials and blinding. This means that it is much harder to perform RCTs on many CAM modalities in the Western paradigm.

Another question that arises around whether EBM is the only arbiter of quality, safety and efficacy is whether the RCT is as applicable to complex interventions as it is to single-substance pharmaceutical interventions. Compared with drug trials or trials of surgical procedures, the design and development of a trial of an integrated intervention or therapy is far more complex. Randomised trials alone may not be able to tell us why an intervention was or was not successful, or which part might have been.

This is not an argument against providing evidence for CAM but is an argument for being more flexible and creative in the way that CAM research is carried out. Even when there is useful evidence available, it is not always easy for doctors to become aware of this evidence or to have it reported in an unbiased way.

A SYSTEMATIC APPROACH TO EVIDENCE

Because of limited time and skills in searching the medical literature, many GPs look to educational seminars and conferences, or up-to-date review articles in general practice or family medicine journals that are tailored to GP management of various conditions. It is assumed that the speaker or author has reviewed the evidence in an objective and unbiased way and is able to clearly summarise it. Such articles and talks can be a simple, helpful and time-efficient way to get an overview of a topic, but there are potential limitations. The material may be presented by a specialist who does not understand the particulars of general practice mentioned above, or it may be less objective and unbiased than we would hope for.

Exploring a clinical question or decision for oneself can be very informative and excellent practice in critical thinking. Box 2.1 gives an outline of a systematic approach to exploring an issue through EBM.

Having skills and knowledge about how and where to find reliable evidence is a prerequisite for the abovementioned approach. (Some resources are listed at the end of this chapter.) It takes time to learn to search the medical literature in an efficient and targeted way. Using the right search words and strategy can mean the difference between a long, fruitless search and a short, fruitful one.

PUBLICATION BIAS

A clinician can only make decisions based on the quality of evidence or advice that is available. We generally trust that when we consult a medical database or the opinion of an expert body we will receive unbiased and objective information. Unfortunately, this is often not the case. All evidence is not equal.

Retail spending on prescription drugs in the USA climbed from $US78.9 billion to $US154.5 billion between 1997 and 2001.⁵ With such amounts at stake, market forces commonly influence research, publication and the interpretation of research data. Thus, even with the best intentions, the basis upon which clinicians make decisions will often be biased as a result. For example, pharmaceutical companies have an interest in promoting antidepressant prescribing. Not only is it advantageous to increase the potential market—for example, prescribing for younger patients—but it is also advantageous to lower the threshold for prescribing. However, although most of the antidepressant trials published show a positive effect, in many of these trials the data have been interpreted in a way that makes the findings look more positive than they really are, and almost none of the negative trials are published.⁶ The actual effect size of antidepressants is more than 30% lower than published trials would have one believe. A fuller analysis of both the published and the unpublished data suggests that antidepressants actually have no more than a placebo effect for mild to moderate depression and only a marginal effect for severe depression,⁷ with approximately 80% of the clinical effect being attributable to the placebo response.⁸ Concerns about the effect of publication bias on other medication categories such as chemotherapy also exist.⁹

For systematic reviews to be meaningful, there needs to be an assessment of all trials, positive and negative, and complete honesty and transparency about findings. Although we like to think that drugs which make it to market have been intensively scrutinised, researchers from the University of California, San Francisco, found troubling evidence of suppression and manipulation of data in studies published in (or often withheld from) peer-reviewed medical journals.¹⁰ They compared the information that companies shared with the USA’s Food and Drug Administration (FDA) about those drugs on application for approval to market, with what was eventually published in medical journals. Only three-quarters of the original trials were ever published, and those with positive outcomes were nearly five times as likely to be published as those that were negative. There may be many reasons for this—commercial reasons for the pharmaceutical companies, professional or academic issues for the researchers who want to be associated with a ‘successful’ drug development, decisions by the editors of peer-reviewed journals who may feel that positive results make better reading and give them headlines in the mainstream press.

The huge investment in getting pharmaceuticals to market also brings with it a reluctance to reveal adverse outcome data, as we saw in the case of Celebrex and Vioxx (the COX 2 inhibitors), as well as the wholesale prescription of hormone replacement therapy (HRT) to perimenopausal and menopausal women prior to the release of the Million Women Study and the Women’s Health Initiative Study showing the relationship between HRT and increased rates of breast cancer and myocardial infarction.

MEDICAL PRACTICE, MARKETING AND EBM

The entanglement of clinical medicine with the pharmaceutical industry is widespread and has almost infinite potential to influence practice.¹¹ Many expert bodies that determine therapeutic guidelines have multiple members with conflicts of interest due to their association with pharmaceutical companies. Sponsors of medical educational seminars and conferences are also known to influence the speaker list and content. Many of the medical stories promoted by the media are industry-driven, although this is rarely transparent to consumers or doctors.¹² Transparency is therefore a vital part of a doctor’s ability to make an informed decision about how much weight to give to a particular guideline or opinion.

Many common medical practices that had been accepted for decades have been found to be contrary to evidence when tested by studies. For example:

Well-supported, safe and effective therapies such as omega-3 fatty acids in the management of hyperlipidaemia, glucosamine for osteoarthritis and exercise for improving cancer survival should be considered first-line therapy options. Rapidity in adopting newly approved pharmaceutical treatments, slowness in rationalising or ceasing use of a conventional therapy or practice that is unfounded, or slowness in adopting an unconventional therapy that is well founded, points to factors other than evidence affecting clinical decisions. These factors could include such things as marketing, peer expectations, medico-legal concerns, political forces, complacency and habit.

There are other CAM modalities for which there is no convincing evidence, such as iridology or Vega machines as diagnostic tools. These may waste resources but be harmless enough unless the advice given about their use misleads the patient or delays important diagnosis and treatment. Patient concerns, fears and expectations can have a significant impact on a doctor’s clinical decisions, for better or worse. For example, in some countries pharmaceutical companies can market drugs directly to patients. Advertising aimed at encouraging patients to request a drug from a doctor will increase many-fold the likelihood that the doctor will prescribe the medication even in the absence of any clinical indication. Similar concerns can arise about the inappropriate use of CAM, particularly where patient fears are greatest, such as in cancer patients.

What lessons can we glean from the abovementioned examples? First, even with the best intentions, standard medical practice is often contrary to guidelines, and guidelines can be contrary to evidence. Sometimes the continued use of unproven therapies can be costly or produce adverse events. Unfortunately, industry rather than good evidence often drives practice, leading to inappropriate practice and over-prescribing. This, combined with a lack of healthy scepticism about therapies, whether conventional or complementary, can have significantly negative effects for doctors, patients and the healthcare system.

‘THERE IS NO EVIDENCE’

When someone such as a medical colleague or authority states that ‘there is no evidence’ for a particular therapy, it is useful to stop and reflect upon the fact that this statement could mean a number of things. It could, for example, mean that:

Healthy scepticism and an enquiring and open mind are vital for the doctor who wishes to practise an informed form of EBM.

RESOURCES

Carefully chosen textbooks and subscription to peer-reviewed journals can help to provide ready access to quality information. For more specific questions or issues, medical and psychological databases can provide published studies and review articles, and some also have consumer databases that are useful to recommend to patients. Learning how to search a medical database efficiently takes practice and some guidance from experienced peers. Some of the most authoritative databases include: