Patients who are ill (and their family members) often want to hear details about other individuals with similar illnesses who have used a treatment and recovered. If the treatment appears to be safe and there is little risk of harm, evidence from these stories frequently appears to be sufficient for them to decide to use the treatment. Patients may interpret this as a sign that the treatment is effective, and to them this evidence is important and relevant for both CAM and conventional medicine. The skilled and informed clinician will need to place this type of evidence into an individual and patient-centred context with respect for all the evidence available from both qualitative and quantitative investigations.

Health care practitioners (conventional doctors, CAM practitioners, nurses, physical therapists) also want to know what the likelihood or probability is that a patient will recover or be harmed based on a series of similar patients who have received that treatment in clinical practice. Such information may come from case series or clinical outcomes studies or RCTs (Guthlin et al., 2004, Witt et al., 2005a and Witt et al., 2005b). They also want to know about the safety, complications, complexity and cost of using the therapy, and this information comes from the collection of careful safety data and health economic analyses.

Clinical scientists will value the same type of evidence as clinicians but will often look at the data differently because of their research training and skills. They often want to know how much improvement occurred in a group who received the treatment compared to another group who did not receive it or a group that received a placebo. If 80% of patients who received a treatment got better, do 60% of similar patients get better just from coming to the doctor? This type of comparative evidence can only come from RCTs, which is the major area of interest for most clinical researchers. These types of studies can include a placebo control but sometimes are pragmatic studies, which compare two treatments or have a non-treatment arm or other types of controls.

Laboratory scientists focus on discovering mechanisms of action. Basic science facilitates understanding of underlying mechanisms and allows for greater precision testing in more highly controlled, (and artificial) environments.

Those in charge of determining public policy often need aggregate ‘proof’ that a practice is safe, effective and cost-effective. This usually involves a health economic perspective within a complex process of treatment evaluation. Systematic reviews, meta-analyses and health services research including randomized trials and outcome studies provide this type of evidence. Health services research also provides data that evaluates the cost, feasibility, utility and safety of delivering treatments within existing delivery systems.

While day-to-day decision-making is more complex than the brief summaries above, the point is that different audiences have legimate evidence needs that cannot be accommodated by a ‘one size fits all’ strategy. The CAM researcher must keep in mind the need for quality research in a variety of domains and attend carefully to the audience and use of the results of their research once collected and interpreted (Callahan, 2002 and Jonas and Callahan, 2002). As stated by Ian Coulter, there is a difference between the academic creation of information in EBM and the clinical application of knowledge in EBP and investigators should keep EBP and the patient perspective in mind when designing and interpreting research (Coulter & Khorsan 2008). In addition, more social science research is needed on models, applications and dynamics of EBM to help guide that interpretation (Mykhalovskiy & Weir 2004).

What are the elements of a research strategy that matches this pluralistic reality? How can we build an evidence base that has both rigour and relevance? In the diverse areas that CAM (or indeed conventional primary care) encompasses, at least six major knowledge domains are relevant. Within these domains are variations that allow for precise exploration of differing aspects of both CAM and conventional health care practice.

In conventional medicine, knowledge domains often follow a hierarchical strategy with sophisticated evidence-based synthesis at its acme (Sackett et al. 1991) (www.cochrane.org). The hierarchical strategy can be graphically depicted by a pyramid (Figure 1.1). At the base of this hierarchy are case series and observational studies. This is then followed by cohort studies in which groups of patients or treatments are observed over time in a natural way, often without any inclusion or exclusion criteria. Randomized studies come next. Here, the decision about which treatments an individual receives is generated by a random allocation algorithm. This usually involves the comparison of two or more treatments with or without a sham or placebo treatment. If several of those studies are then pooled they produce a meta-analysis. This is a summary of the true effect size of an intervention against control.

There are several alternative strategies to the evidence hierarchy that attempt to balance the risk of the poor relevance it produces. One of these is called the ‘evidence house’ and seeks to lateralize the main knowledge domains in science in order to highlight their purposes. It does this by aligning methodologies that isolate specific effects (those with high internal validity potential) and those that seek to explore utility of practices in real clinical settings (those with high external validity potential) (Figure 1.2). For example, mechanisms, attribution and research synthesis each seeks to isolate specific effects whereas utility is investigated though methods that assess meaning, association and cost (both financial cost and safety). Each of these knowledge domains has its own goals, methodology and quality criteria. The corresponding methods for each of the six domains are: for isolating effects, laboratory techniques, RCTs, meta-analysis; and for utility testing, qualitative research methods, observational methods, health services (including economic) research. Of course, variations in these methods can often be mixed in single studies, producing ‘mixed-methods’ research (such as qualitative studies nested inside RCTs) that seek to address dual goals. This strategy can be conceptualized in what has been called an ‘evidence house’ (Jonas 2001; Jonas, 2005). In this strategy the knowledge domains are placed in relationship to each other and the primary audience they serve. The evidence house helps balance the misalignment produced by the evidence hierarchy, by linking the method to the goal to the primary audience. The six major knowledge domains of the evidence house are briefly described below.

The circular model explores the relationship of the clinical methods used in the middle two domains of the evidence house. It assumes that there is no such thing as ‘an entirely true effect size’ but that the effect sizes vary based on patient recruitment, specific therapists (Kim et al. 2006) and the environment (context) in which that therapy is provided (Hyland et al. 2007). This suggests that we may have difficulty in completely controlling for bias and confounding when we have no real understanding of the underlying mechanisms of the treatments being delivered. In these circumstances further development of a circular model may allow us to arrive at an approximate estimate of reality as it relates to complex pictures of chronic disease within the community (Walach et al. 2006) (Figure 1.3).

In a circular strategy the principle is that information from all sources is used to establish consensus around the most appropriate therapeutic approaches in a particular therapeutic environment. This allows development of best practices even when there is little RCT evidence. We have to bear in mind that clinicians are obliged to treat people even when they present with conditions for which the treatment is not supported by a substantial body of research (Flower & Lewith 2007). The circular strategy also actively involves patients in the decision-making process, which could be important since there is evidence that empowering patient decisions has an impact on outcome (Kalauokalani et al., 2001, Bausell et al., 2005 and Linde et al., 2007a). Acupuncture, as practised within a western European medical environment, is a prime example of this, as the debate about the evidence from both the German Acupuncture Trials (GERAC) and Acupuncture Research Trial (ART) studies illustrates (Linde et al., 2005, Melchart et al., 2005, Witt et al., 2005a, Witt et al., 2005c, Witt et al., 2006, Brinkhaus et al., 2006 and Scharf et al., 2006).

By and large most complementary treatments are widely available, have often been in use for a long time and in some countries even have a special legal status. Consequently we may wish (for reasons of public health and pragmatism) to evaluate the safety of the intervention and the quality of the practitioners providing that intervention, before conducting research on theoretical or specific biochemical mechanisms that may be triggered by a particular product or practice. These strategic differences between the research approaches that may need to be applied to conventional and complementary medicine are summarized by Fonnebo et al. (2007) (Figure 1.4).

For example, a large pharmaceutical company will develop a new pharmaceutical through a phased approach that will involve both in vivo and in vitro laboratory experiments before preliminary human evaluation of a new chemical agent. This contrasts dramatically with the evaluation of a complementary medical intervention such as homeopathy and acupuncture, where opinions and beliefs about its veracity, effect and safety are widely debated and diverse. Inevitably this influences the public’s expectation and opinion about a particular intervention which may have an impact on any systematic evaluation of the therapy and its equipoise within a clinical trial.

New, more sophisticated and pluralistic approaches to EBM incorporating the full spectrum of the information needed for making clinical decisions are being developed for all medical interventions; examples include the RAND ‘appropriateness’ approaches (Coulter et al. 1995), the Agency for Health Care Research and Quality’s (AHRQ) efforts on consumer or patient-centred evidence evaluations (Clancy and Cronin, 2005 and Nilsen et al., 2006), decision models and new ‘synthesis’ approaches (Haynes 2006) and ‘care pathway’ applications of EBM (Astin et al. 2006), as well as the comparative effectiveness research by the Institute of Medicine (Sox 2009). All of these approaches have their own strengths and weaknesses and call for systematic incorporation of concepts such as goals, problem formulation and values into the formulation of consistent and customized EBM decisions that account for the complexity of information needed in the clinical setting. Taken together, it might be worthwhile distinguishing some of the more academic debates around EBM from the practical applications by defining EBP (Coulter & Khorsan 2008). We believe that EBP requires, at a minimum, the establishment of quality standards for each of the information domains described in Figures 1.2 and 1.3 and may be thought of as ‘best research guidelines’.

In addition to internal and external validity, previously described, CAM research also requires evaluation of ‘model validity’, which assesses the likelihood that the research has adequately addressed the unique theory and therapeutic context of the CAM system being evaluated (Jonas & Linde 2002). Many CAM systems arise outside developed countries or western theories so cross-tradition and cross-disciplinary research, involving experts in both conventional medicine and CAM, is often required (Jonas 1997). Some standardized treatments can produce marked variations that may be culture-specific; consequently results from one cultural environment may not always translate to another, particularly with respect to differences in informed consent (Bergmann et al., 1994, Kirsch, 1999 and Moerman, 2000). Model validity refers to the fact that the research must be a fair exemplar of the practice in use and the method chosen to test it must reflect the goals and context of the audience that will use the consequent information.

Table 1.2 shows a checklist of the main quality criteria for evaluating internal, external and model validity. For convenience, these criteria have been drawn from a variety of quality rating scales (including Scottish Intercollegiate Guidelines Network (SIGN) 50, CONSORT, Cochrane and others) and are collated and summarized in an approach called the Likelihood of Validity Evaluation (LOVE) system, which has been applied to the evaluation of several areas of CAM research (Jonas & Linde 2002). Section 1 addresses internal validity and includes criteria such as randomization, baseline comparability, change of intervention, blinding, outcomes, statistical analysis and sample issues. Section 2 addresses external validity and includes criteria such as generalizability, reproducibility, clinical significance, therapeutic interference and outcomes. Section 3 addresses model validity and includes criteria especially important to whole-systems research. This includes representativeness, informed consent, methods matching, model congruity and context. Finally, Section 4 addresses reporting clarity that includes how well an article or report accurately describes the study.

Model validity criteria highlight a number of conceptual and contextual issues of which the investigator should be aware of when conducting research on CAM (Levin et al., 1997, Vickers et al., 1997, Eskinaski, 1998 and Egger and Davey, 2000). CAM interventions are complex and in general do not consist of a single isolated entity; this is very similar to many of the interventions in primary care (general practice) or surgery (Gabbay & le May 2004). For example, acupuncture is a treatment within TCM and is itself highly variable. Acupuncture has often been considered a single intervention, but evidence suggests that the treatment of addictions such as smoking withdrawal may be employing entirely different neurophysiological mechanisms to those involved in the treatment of chemotherapy-related nausea and vomiting (Lewith & Vincent 1995). As a consequence, trial design may need to be entirely different for acupuncture when applying this intervention in differing clinical contexts with different underlying mechanisms. It may be appropriate to use sham acupuncture when the underlying mechanism is point-specific (as in the treatment of nausea) but entirely inappropriate to depend on point specificity for the design of the placebo when the underlying mechanisms are themselves not point-specific, as in the treatment of addiction.

Modern clinical trial methodology tends to make specific assumptions which may not apply to all aspects of CAM research. Therefore our methodology needs to be applied with an understanding of the science and the CAM intervention involved, and underpinned by theoretical ideas about the underlying mechanisms involved in the intervention. This will allow us to consider how to design research properly with appropriate controls that fit the CAM ‘model’ under study; in other words, with a high degree of ‘model validity’.

While research that maximizes internal, external and model validity is the ideal, the realities of the CAM research environment often make the ideal difficult to achieve. First, funding levels for CAM are quite small compared to research in conventional medicine. Thus, large, multicentred pragmatic or three-armed studies (in which both placebo and standard care comparison groups exist) are few and far between. Research on mechanisms with adequate product quality are unlikely to be supported, such as those required for multicomponent herbal products. The sceptical attitudes to CAM in many conventional circles, the competitive nature of new products and practices and the low profit potential of many CAM products and practices keep CAM research off the priority list for many governments and private funders. Rarely is there a critical mass of experienced researchers for conducting complex studies or mentors and a clear career path for training CAM investigators. Thus, CAM research continues to produce a continual stream of pilot and small studies or larger studies that address rather focused questions providing incremental information. In the context of so many questions about CAM practices, it is not surprising that the public often has no evidence for making decisions about CAM practices.

CAM has been very much the poor relation of research in the context of the EU, UK and US environments. The current UK CAM research budget is 0.0085% of medical research spending (Lewith 2007). When placed against the context of CAM use in the UK (15% per annum and 50% lifetime use), this is hardly an adequate distribution of research funding as far as the population’s use of CAM is concerned. While the total expenditure in the US is substantially larger, it is still miniscule in proportion to CAM use. Consequently there is very limited research evidence within the whole field. Oxman (1994), one of the fathers of EBM, makes it very clear that the absence of evidence of effectiveness should not be interpreted as being synonymous with the absence of clinical effect. A pervasive problem in CAM is the inappropriate interpretation of limited and inadequate data, an approach which substantially misrepresents the underlying priniciples of rigorous EBM.

The first step in selecting an appropriate research method is clearly defining what evidence is needed, particularly in relationship to how it will be used (Feinstein 1989). Each research method has its own purpose, value and limitations. To summarize, when defining concepts, constructs and terms and when assessing the relevance of an outcome measurement, qualitative methods, with their indepth interviews and content analysis, are the most appropriate approach. When seeking associations of variables, surveys, cross-sectional or longitudinal studies with methods that allow for factor analysis, regression analysis or a combination of both, such as structural equation modelling, are the most useful. When trying to measure the overall impact of a complex intervention delivered in a clinical context, pragmatic trials and outcomes methods should be applied. When attempting to isolate and prove specific theoretical effects of treatment on selected outcomes, or determine the relative merit of whole-practice systems, placebo-controlled or pragmatic RCTs are the method of choice, respectively.

The research question, main audience and the utility of a research project will anchor the methodology (Figure 1.5). For example, in multimodality practices that are often not well described (e.g. spiritual healing, lifestyle therapies) and where the interest is on impact on chronic disease, outcomes research or pilot trials are the best initial approach (domain 3A in Figure 1.5). In well-described modalities that are safe and not expensive where effectiveness (not efficacy) may be the main interest (e.g. acupuncture, homeopathy), outcomes data coupled with decision analysis may provide the best strategic approach (domain 3B in Figure 1.5) (Dowie 1996). For many natural products, where the active or standard constituents are variable, basic science (e.g. laboratory characterization and safety data) is needed prior to clinical research (domain 2 in Figure 1.5). Products that are well characterized but have potential direct adverse effects need placebo-controlled or standard comparison RCTs provided their public health implications warrant such an investment. Placebo studies of natural products are more useful for making regulatory decisions (e.g. product marketing and claims) than for individual clinical decisions (domain 4A in Figure 1.5). The US National Institutes of Health’s study of the efficacy of St John’s wort for depression, using a large, three-armed, multicentred, placebo-controlled trial, is an example of this latter approach (Shelton et al., 2001 and Hypericum Depression Trial Study Group, 2002). It is a widely used treatment, has significant potential drug–herb interactions (through the P450 system) and is treating an important, common clinical condition (domain 2 in Figure 1.5).

A physician considering referral of a patient with back pain for acupuncture will want to know what kind of patients local acupuncturists see, how they are treated, whether these patients are satisfied with treatment and their outcomes. This type of data comes from practice audit (observational research), surveys and qualitative research (domains 3B and 1 in Figure 1.5) (Cassidy, 1998a and Cassidy, 1998b). For these situations, information from local practice audits may be more valuable than relying on the results of small (or large) placebo-controlled trials done in settings where the practitioners and populations may be quite different. Data collection, monitoring and interpretation of observational studies must be as carefully performed as experimental studies. Finally, when exploring theories and data that do not fit into current assumptions about causality (e.g. energy healing or homeopathy), a carefully thought-out basic science strategy is needed. Given the public interest and the implications for science of these areas, it is irresponsible for the scientific community to ignore them completely (Jonas 1997).

Within RCTs (domain 4 in Figure 1.5) there are several goals that determine which design and sequence of studies are most appropriate. If the goal is obtaining information on the specificity of molecular or procedural effects (e.g. drug effects), placebo controls are required (domain 4A in Figure 1.5). Placebo controls cannot provide information about added value from a therapy (domain 4B in Figure 1.5), which requires a no treatment or standard treatment control (e.g. hypericum versus antidepressant control). Research summaries (domain 5 in Figure 1.5) and generalizability (domain 6 in Figure 1.5) require systematic reviews and health technology assessment, respectively. Examples of these two are the National Institutes of Health Consensus Conference statement on acupuncture (1997), which influenced public decisions on reimbursement for acupuncture. A similar approach used by the former Agency for Health Care Policy and Research’s practice guideline on low-back pain and manipulation and the subsequent establishment of chiropractic services in the USA (Bigos & Bowyer 1994).

Use of this decision tree, along with quality standards for each of the research methods listed, provides a rational strategy for selecting the most appropriate type of evidence, assures quality for each method and so combines rigour and relevance for creating EBP in a pluralistic health care environment.

Recruitment for clinical trials underpins their generalizability. We have previously discussed the tension between internal and external validity and their relevance to recruitment in any study. CONSORT flow charts identify the number of individuals approached for a study and consequently give a percentage of the number recruited. A good example of this problem in conventional medicine is the evaluation of Antabuse, a pharmacological treatment that makes alcoholics vomit if they continue to take the medication daily. Trials evaluating Antabuse suggested that it was very effective; however, only 9% of the alcoholics approached agreed to be recruited, making it difficult to generalize about the effect of Antabuse in the broad population (Howard et al. 1990). All studies should explicitly consider population selection and breath of application.

When groups are defind as ‘homogeneous’ conventionally and recruited to a clinical trial they may not be homogeneous when evaluated from the perspective of another medical tradition. The differing diagnostic taxonomies of CAM systems may need to be addressed within a research context. The diagnosis of osteoarthritis may represent over a dozen classifications when evaluated by TCM. A ‘standardized’ treatment may approximate the ‘average’ syndrome and this simplifies the treatment strategy. However, this standardized treatment may be suboptimal and so risk producing ‘false-negative’ results within the trial – a sign of poor model validity. Subjects with irritable-bowel syndrome treated with individualized TCM herbs improved for longer than those given a standardized approach and both groups did better than the placebo group (Bensoussan & Menzies 1998). A three-arm study design allows for the evaluation of this aspect of model validity.

In addition to a three-armed trial, a double selection design can be used, in which group selection is done according to both conventional medicine and the alternative whole system. Shipley & Berry (1983) tested the homeopathic remedy Rhus toxicum on a conventinally homogeneous group of osteoarthritis patients without individual homeopathic classification and reported no effect over placebo. Fisher et al. (1989), in a follow-up study, used a double selection approach that provided both good internal validity and model validity. Patients met criteria for fibromyalgia and Rhus tox, producing a group ‘homogeneous’ for both medical systems. In this study those treated with homeopathic Rhus tox did better than those given placebo.

Pilot data should always be obtained to ensure adequate treatment is being tested in a clinical trial. This is frequently omitted. For example, a study of acupuncture for human immunodeficiency virus (HIV)-associated peripheral neuropathy published in the Journal of the American Medical Association reported negative results. Many acupuncturists report good results in this condition (Shlay et al. 1998), but use considerably more treatments than in the Journal of the American Medical Association study. The treatment being tested must be optimal or at least representative for a defined group of providers and piloted for that condition. This is certainly not the case for asthma treated with acupuncture, as only two trials used the same points (Linde et al. 1996).

We would suggest the combination of a detailed literature review and an expert panel approach, designed to build consensus around best practices (Ezzo et al., 2000 and Flower and Lewith, 2007).

It is typical to see 70–80% effectiveness reported from open practices in both conventional and alternative medicine (Roberts et al., 1993, Jonas, 1994, Guthlin et al., 2004 and Witt et al., 2005b). Traditional medicine systems, various mind–body approaches and psychological and spiritual systems often attempt to induce self-healing by manipulating the MAC effects on illness; MAC is often confused with placebo, which itself interacts with non-placebo elements of therapy in complex ways (Bergmann et al., 1994, de Craen et al., 1999 and Moerman, 2002). Arthur Kleinman’s classic study on why healers heal illustrates the importance of these contextual issues (Kleinman et al., 1978 and Moerman and Jonas, 2002).

Outcomes that are objective and easy to measure, or well known, are often selected instead of more difficult patient-oriented, subjective outcomes, though the latter may be more relevant. Eric Cassell (1982) has pointed out that, because of an overemphasis on finding a ‘cure’, researchers may seek more objective, disease-oriented outcomes that move away from gaining knowledge that may help in the more subjective, healing process of the illness, a core goal of medicine. The investigator, in the name of ‘rigour’, may try to maximize internal validity by using objective measures instead of measures that are meaningful to patients. For example, many physicians deem ‘pain relief’ a fuzzy, non-objective outcome. However, for patients suffering from pain, its relief is the only really relevant outcome.

While non-patient-centred outcomes selection is not a unique issue for CAM it may be one of the main reasons for CAM’s popularity. Cassidy, 1998a and Cassidy, 1998b conducted interviews with over 400 patients visiting acupuncturists. Most patients did not experience a ‘cure’ but they continued acupuncture treatment because of other factors such as an improved ability to ‘manage’ their illness and a sense of well-being. Measuring these outcomes may require including qualitative research methods, or developing and validating new measurement instruments. Researchers may settle for popular outcome measures, such as the SF-36 (Guthlin & Walach 2007), while forgoing more individualized outcome measures, such as the Measure Your own Medical Outcome Profile (MYMOP) (Paterson, 1996 and Paterson and Britten, 2003). Recently, a database of CAM-relevant outcome measures has been developed and posted online: IN-CAM Outcomes Database (www.outcomesdatabase.org).

Hypothesis-focused research allows investigators to identify cause-and-effect relationships. This is a powerful method for confirming or refuting theories about treatment–outcome associations. But by focusing on a particular theory-driven causal link we risk ‘fixing’ our conclusions about the value of a therapy, which in turn may restrict the investigation of other, possibly better approaches (Heron 1986). Given the small resources invested in CAM research, potentially superior CAM practices may never see the light of evidence, compared to better-funded and profitable treatments.

Once treatments are established they become the ‘standard of care’. After this, alternative treatments are ethically more difficult to investigate. Lifestyle therapy for coronary artery disease (CAD) is an example. Current standard-of-care treatments for CAD are oriented around an anatomically based hypothesis of CAD. Coronary artery bypass grafting (CABG), balloon angiography and stents are all established treatments of CAD. An extensive industry has developed around these procedures and with it the economic and political interests to sustain these treatments. Lifestyle therapy can also successfully treat CAD but is based on a non-anatomical hypothesis of CAD aetiology not tied to a reimbursement and other industry drivers. It may be that lifestyle therapy is better than CABG for treatment of CAD in certain populations given its low cost, preventive potential, reduction in fatigue and improvement in well-being (Blumenthal and Levenson, 1987, Haskell et al., 1994, McDougall, 1995 and Ornish et al., 1998), but this is difficult to investigate as it is unethical to mount placebo-controlled trials in this life-threatening situation. Revealing partial causes runs the risk of preventing the discovery of potentially more beneficial therapies based on other hypotheses. Research strategies for chronic disease need to have the flexibility to test multiple hypotheses (Horwitz, 1987, Coffey, 1998 and Gallagher and Appenzeller, 1999).

When the results of observational and randomized studies are different it is usually assumed the randomized study is more rigorous and valid. This assumption has been challenged by an increasing number of studies showing that, properly done, observational studies may not differ in outcome from those that are randomized (Benson and Hartz, 2000, Concato et al., 2000 and Linde et al., 2007b). This does not in any way invalidate RCTs but rather suggests that much useful and important data may be collected utilizing a variety of different methods. Some practitioners of complex traditional or alternative medical systems believe that randomization can interfere with a therapy (by eliminating choice), obscure our awareness and so bias outcome. For example, TCM is based not on direct cause-and-effect assumptions but on an assumption that ‘correspondences’ occur between system levels (biological, social, psychological, environmental) (Porkert 1974). An assumption in TCM theory is that if a practice influences one level it can affect all others. Thus, finding direct cause-and-effect links may be less important for a TCM practitioner than to establish accurate correspondences between the body and season (Unshuld 1985). From this perspective extensive efforts to establish isolated causal links between intervention components and outcome (as done in non-pragmatic RCTs) are of less interest than in western medicine (Lao 1999).

Blinding or masking is primarily used to control for expectation effects in clinical studies and to reduce assessment bias. Although there is no foolproof method to ensure successful blinding, it is still an important part of quality research (Begg et al. 1996). Some alternative treatments, such as certain herbal preparations and superficial acupuncture techniques, can be at least partially blinded. Special attention is needed to match smell and taste in herbal products. When giving acupuncture, the practitioner cannot be completely blinded, except with special types of acupoint stimulation, such as with ultrasound or laser. Homeopathy or healing is more easily double-blinded than most conventional drugs but complex behavioural or lifestyle programmes cannot be delivered blind. Controlling for time and attention can often be achieved and it is possible to code groups and so do blind measurement of outcomes and analysis. For example, equal time and the presence of a sham therapist can be incorporated into studies of therapeutic touch (Gordon et al. 1998). In most traditional medical systems such as Ayurvedic or Native American medicine, a deep connectivity to all information about a therapy is assumed to coexist and complete blinding is not believed ever to be achievable. These systems assume that expectation effects are largely unconscious (Rigby 1988). In fact, a therapy in a traditional system often tries to enhance these unconscious expectation effects by manipulation of ‘spirits’ or ‘energy’ and the patient’s belief system associated with them (Jonas 1999). From this perspective masking and sham interventions simply eliminate our awareness of the therapeutic agent, but not the agent itself, however implausible we may think this to be. There is evidence that autonomic and physiological effects occur unconsciously (such as reactions to music when under anaesthesia), something that many traditional medical systems would predict (Bennett, 1986, Blankfeild, 1991 and Radin, 1997). Unconscious expectancies may influence outcomes in clinical trials. For example, the physiological effects of ‘active’ placebos (drugs that produce ‘side-effects’ but not the therapeutic effect) in studies of antidepressants can influence outcomes even within double-blind studies (Greenberg et al., 1992 and Kirsch and Sapirstein, 1998). These assumptions should not be used as an excuse for conducting or accepting poor-quality research but they do make a case for non-experimental (observational) studies and non-blind studies.

Many CAM therapies involve teaching and learning, such as biofeedback, meditation, imagery and hypnosis. The skill and ability of the teacher (therapist) and the student (patient) may affect outcome (Kim et al. 2006). Blinding in such clinical trials is not usually possible and if it were, it would reduce the very processes upon which the effects of the intervention are thought to be based. Some physiological reactions may change, even during a single therapeutic episode, and so the degree of blindness may also change, altering equipoise, expectation effects and the relative efficacy of the ‘active’ treatment. The goal of many of these therapies is to increase the patient’s awareness of unconscious processes and enhance self-efficacy.

Preferences have important consequences for clinical research, especially in CAM where patient preferences and belief may be high. Randomization is ethical only when equipoise exists between treatment groups (Black 1996). This can occur only when efficacy is truly ambiguous and there are no strong preferences for any therapy. Equipoise is present by definition when new drugs are tested, especially if no established therapy exists, and placebo controls are then ethically acceptable. This situation often does not exist in CAM. If patients are using or avoiding CAM therapies, it is often because of strong preferences and beliefs which may influence outcomes (Furnham and Kirkcaldy, 1996 and Fonnebo et al., 2007).

Recruitment to RCTs is problematic in these cases and good clinical studies have either been abandoned or the process of randomization dropped because of the refusal of patients to enter such studies (von Rohr et al., 2000 and Katz et al., 2005). Randomization processes in conventional studies have been subverted by well-meaning health care practitioners, a problem that may also happen in CAM research (Schulz et al. 1995). In addition, research may be contaminated when subjects use alternative therapies that interact with the treatments being tested. For example, over 16% of patients in clinical trials at the US National Institutes of Health Clinical Center were using herbal therapies and not reporting this to the investigators (Sparber et al. 2000). The opposite trend might also be a problem: if patients on a trial for a CAM intervention take conventional drugs in addition to the CAM treatment, and do so differently in the control or treatment group, the outcome may be biased.

Most therapies become used and often accepted before good evidence on their efficacy is available (Eddy 1990). Acupuncture provides a current CAM example: there is a growing acceptance of acupuncture in general, even in the absence of proof for its effectiveness for most conditions. The clear safety of acupuncture (Melchart et al. 2004) makes many clinicians and researchers willing to recommend it with less evidence than other higher-risk interventions. This type of risk stratification is a useful step in deciding what type of evidence is needed for balancing rigour, relevance and cost in EBP. For many therapies and conditions observational research may be initially sufficient for clinical decisions about best practices. For example, few patients or physicians would recommend against the use of a low-cost and inexpensive therapy (e.g. mind–body or homeopathic treatment) for a non-life-threatening disease (e.g. seasonal allergies) if outcomes data indicated that it was safe and benefited patients (Linde & Jonas 1999).