We plan to discuss the principles and concepts that underpin randomized controlled trials (RCTs). This research methodology can be employed in a number of different contexts but RCTs are usually used in clinical settings. A trial, holding other factors constant, can be run to compare performance about a number of criteria, such as: ‘which treatment works better than a placebo?’ or ‘which device is most energy-efficient?’ The fundamental principles of RCTs remain the same in whichever context they are applied and this chapter aims to outline the steps that are important in setting up an RCT and meeting the requirements of a research protocol. RCTs allow us to answer very specific questions. They set out to evaluate the effect of a particular treatment or strategy in a population where an intervention is introduced, often by comparing the outcome with a control group where no intervention, or a standard intervention, may have been used. The population must be well defined and carefully selected.

A number of implicit assumptions underpin the RCT as follows (adapted from Vickers et al. 1997):

These tenets lead to the claim that the RCT provides the ’gold standard’ for research. It is the best means of attributing real cause and effect and therefore adds to our stock of knowledge. Not all researchers necessarily believe all of these, but they provide a sound framework for conducting a RCT.

In effect RCTs are mainly used to test a predefined hypothesis. When planning a confirmatory trial it is helpful to do a pilot trial with the aim of generating a hypothesis and to test the planned outcome measures and the study protocol for feasibility. In addition, pilot studies can be helpful in providing an idea of the effect size of the intervention and this in turn will inform the sample size for any larger, more definitive study. In these pilot trials statistics are used on an exploratory basis. The present published literature suggests that in CAM research confirmatory studies are often done without pilot trials and quite a large proportion of the confirmatory (definitive) studies seem like pilot studies and are too small to have enough power to detect a significant difference between groups. The reasons for this might be limited financial resources for CAM research and the fact that CAM has fewer qualified and experienced researchers than conventional medical research environments.

When planning a study it is important to clarify the study in more detail by considering the following aspects (Chow & Liu 2004):

Developing a hypothesis is an important issue when planning these studies. A hypothesis always consists of a null hypothesis (H₀) which assumes no effect and an alternative hypothesis (H_A) which holds the null hypothesis not to be true and consequently assumes an effect. The alternative hypothesis is more directly connected to the specific research question.

The aim of confirmatory studies is to answer research questions based on the hypothesis proposed by the researcher. Therefore, posing an adequate and answerable research question is essential as a clear, well-researched, thoughtful and specific question based on appropriate study design has a reasonable chance of providing a valid answer. One of the main reasons (other than failure to recruit) why the majority of research proposals are either unclear or fail to provide a useful answer is that the initial research question itself lacks clarity. The core elements of a precise research question for a confirmatory RCT can be summarized by PICO (patients, intervention, control intervention, outcome = primary endpoint; see example in Figure 5.1). Using PICO can be very helpful in developing your hypothesis.

The research question and the subsequent hypothesis build the basis for hypothesis testing. If the null hypothesis can be rejected based on a predefined significance level (e.g. 5% or 1%), the alternative hypothesis will be accepted. This means that, from the example shown in Figure 5.1, if there is a significant difference on the visual analogue scale (P < 0.05) between the acupuncture and the diclofenac group, the null hypothesis (H₀: acupuncture = diclofenac) can be rejected and the alternative hypothesis (H_A: acupuncture ≠ diclofenac) is applicable.

If there is no significant difference between both groups this does not necessarily mean that the null hypothesis is true as there may simply not be enough evidence to reject it. For example, if the sample size is too small the study may be underpowered. The chance of the study providing a statistically significant outcome is based on assuming that a 5% (or less) chance of this event occurring randomly is significant. This could mean that, with a P-value of 0.04 (a 4% chance of this happening randomly), the null hypothesis would be rejected, whereas with a P-value of 0.06 (a 6% chance of this happening randomly) there would not be enough evidence to reject the null hypothesis. The choice of a significance level of 5% is arbitrary but applicable throughout biological science and therefore the outcome of the study needs to be interpreted with caution, particularly with respect to the number of people entered. The greater sample size or number of people entered into the study, the more statistical ‘power’ it has and the more its statistical conclusions can be ‘trusted’.

However a significant difference between groups does not relate to the clinical importance of this finding as the statistical significance depends to a large extent on the sample size (number of people) and the variability of the condition within the study. Accordingly a large study might find a small but clinically unimportant difference between treatments, which is highly significant. Equally, a small study with few people might find a large difference that could be clinically important. Clinical importance describes a difference between two treatments that has a relevant effect size which is noticeable for, and valuable to, a patient with that condition.

Most randomized controlled studies evaluate if one intervention is superior to another. This is generally the case for treatment comparisons with a waiting list or a placebo intervention as controls. However, for some comparisons we test for similarity between two treatments (equivalence) or non- inferiority, but these equivalence studies, which include both non-inferiority and non-superiority trials, are rare in CAM. A non-inferiority hypothesis could, however, be very useful when comparing a CAM treatment with a conventional treatment. Non-inferiority or equivalence trials have methodological features that differ from superiority trials. In an equivalence trial, the null and alternative hypotheses are reversed. This means the null hypothesis is that treatment A is different from treatment B, whereas the alternative hypothesis is that there is no difference between the treatments.

In addition the effect size of the reference treatment has to be known and the margin for the non-inferiority has to be predefined for the power calculation and subsequent sample size. This margin represents the smallest value for a clinically relevant effect and outcomes within this margin are defined as non-inferior (Piaggio et al. 2006). The interpretation of the results depends on where the confidence interval lies relative to both the margin of non-inferiority and the null effect. For two-sided evidence (equivalence trials) two margins are needed (one above zero and one below) (Figure 5.2).

When planning a study probably the most important question is how many patients are needed to have enough statistical power to detect a clinically meaningful difference between the treatment groups. If the budget is limited or the medical conditions allow only a small number of patients to be included in the study, the study may fail to reach adequate power and could be of very limited value. An appropriate sample size is essential for a worthwhile study. The sample size calculation requires that the hypothesis should be clearly stated and a valid design and appropriate test statistics should be used. In hypothesis testing two errors can occur: the type I error (α) takes place when the null hypothesis is rejected although it was true and the type II error (β) is when the null hypothesis is not rejected when it was false. The probability of making a type I error is called the level of significance whereas the type II error is called power. A type I error is usually considered to be the most important of the two items this approach primarily tries to control for. To determine the sample size the investigator should provide the following information:

In clinical studies a significance level of 5% is usually chosen to reflect 95% confidence in the conclusions and usually a power of 80% or 90% is used in conjunction with this. Estimating a clinically meaningful difference and the standard deviation for the planned trial is often difficult, especially for CAM treatments, where there may be no previous data for the outcome for the intervention being evaluated. If the expected difference is large, fewer patients will be needed than if the expected difference is small. An important influence on the sample size is the variability of the patient outcomes for the primary outcome. If a high standard deviation is expected (lots of variability) more patients are needed to detect a significant difference between treatments than if the patient outcomes are more homogeneous.

Many CAM studies are conducted with small research budgets and consequently small sample sizes; therefore the sample size which would detect a significant difference between treatment groups may not be achieved. From a methodological perspective it is bad science to perform an underpowered study. If the budget is not sufficient to perform a trial of adequate size it is better to do no study, unless it is a pilot.

The RCT is the most reliable study design to detect whether the difference in the outcomes of two or more interventions is caused by a specific treatment. In effect it is the ‘gold standard’ for clinical research. With the aim for control for baseline differences, patients are allocated at random to receive one of several interventions. In addition, the treatment group is compared to at least one control group. Different control groups exist and the choice of control depends on the research question (Table 5.2). Placebo-controlled trials only tell us if the treatment is better than a placebo and we always need more information that that to inform clinical practice, where many treatments may be available for one condition.

Randomized trials can be open or blinded but it depends on the specific intervention as to which activities within a trial can be realistically blinded. Although RCTs are relatively straightforward, a lot of methodological errors have occurred in the past and no doubt will continue to appear in future literature. Guidelines for performing RCTs have been developed to improve their quality. These include good clinical practice, which regulates the functions and processes or the reporting (Altman et al. 2001) in RCTs as well as the inclusion of a standard reporting system for trial recruitment and compliance (Moher et al. 2001). Some of the CAMs that may be evaluated also have specific reporting requirements that are associated with Consolidated Standards of Reporting Trials (CONSORT); this includes Standards for Reporting Interventions in Controlled Trials of Acupuncture (STRICTA) (MacPherson et al. 2001) and for homeopathy RedHot (Dean 2006). These standards are of great importance and improve the quality of RCTs but they require more resources and funding in order to perform the RCT.

Randomization is usually used to control conscious and unconscious bias as well as any unknown confounders in the allocation of patients to treatment groups. The purpose is to generate comparable patient groups with similar baseline characteristics. It is vital that the random allocation is performed properly and described in detail in the methodology, as inadequate or insecure randomization can be associated with very biased and usually positive findings (Shulz et al. 1995).

There are many examples where randomization is insufficiently performed and reported (Altman et al. 2001).

In order to avoid this criticism in planning a study the following issues need to be considered:

For instance, cluster randomization is used when whole social groups (families) or GP practices have to be assigned to different intervention groups. However, most trials use patient-level randomization and assignment is performed independently for each patient. In a two-armed trial with a 1:1 ratio the chance of a patient receiving either verum or control is 50%. RCTs with more arms (e.g. verum, control 1 and control 2) can also involve equal numbers in each group; for example in a three-armed trial with a 1:1:1 ratio the probability to be assigned to one of the groups is 33%. Unequal randomization with a higher chance of being assigned to the verum group can be helpful in an environment where the acceptance of the randomization process is poor from the patient’s or practitioner’s perspective. Patients with an interest in CAM often have clear preferences for CAM and this can be a major influence on their decision to participate in a randomized study. Therefore a 2:1 opportunity to receive their preferred treatment could improve the acceptability of randomization and improve recruitment and retention for the study. Ethics (e.g. in cancer) can also be an argument for unequal randomization. However, unequal randomization has disadvantages as the study may lose statistical power and blocking can be compromised, so it should only be used if there are very important reasons for doing so.

Simple or complete randomization is performed when a single random list is used and each patient is randomized regardless of the recruiting study centre and patient characteristics. In small trials or multicentre trials this can result in imbalances of group size and patient characteristics between the groups, but procedures for managing and restricting randomization help to solve this problem. The most frequently used methods are blocking and stratification. Both can be predefined and taken into account when generating the randomization list. Block randomization is used to ensure that the numbers of patients in the groups are comparable. In a two-armed trial with a randomization ratio of 1:1 and a block size of eight, four patients will be randomized into the verum and the other four into the control group. One important problem of blocked randomization is that allocation becomes predictable if the size of the blocks is known by the person who assigns and treats the patient, so this information must remain secure. This can be solved either by variable block size or blinding the study personnel to the block size. Stratified randomization is used to ensure that groups are comparable for important characteristics such as the provider of the treatment or the disease stage. This is done by doing separate randomization for ‘strata’ such as study centres or asthma staging. Stratification requires blocking within the ‘strata’, otherwise it too can become ineffective (Altman et al. 2001). Blocking and randomization should be used carefully. Figure 5.3 provides an example for stratification of study centres and using a variable block design.

One can adjust the probability of assignment of new patients to treatments during the study (Chow & Liu 2004). This is called adaptive randomization. Patient outcomes can be used as they become available to adjust the randomization assignment for future patients, perhaps assigning more of them to the ‘better, more effective treatment’. This allows one to improve expected patient outcomes during the experiment, while still being able to utilize the best available statistical decisions in a timely fashion. This type of randomization does not require the randomization list to be generated before the study starts. The random assignment is generated directly when the current patient is randomized, usually by a computer-based algorithm.

There are many ways to generate a random list prior to study start (Jadad & Enkin 2007). Older methods such as flipping a coin or throwing dice have been replaced by a number of validated software packages (e.g. SPSS, SAS, STATA) which ensure the quality and rigour of the random sequence.

Empirical investigations have shown that concealment is the most critical element of randomization (Schulz 1995). It is absolutely essential to ensure a concealed randomization procedure or the study loses rigour. The study personnel must not be able to predict the group to which a participant will be allocated and they should not be in a position to modify this allocation. The ideal strategy for concealment depends on the intervention. In drug trials identically packaged, consecutively numbered drug containers can be used based on the randomization list. For other interventions, central randomization, using a telephone hotline or an online database, is suitable. The physician who includes a patient contacts an external clinical trial unit by phone or via the web and they inform the physician about the group allocation if the trial is single-blind. In a double-blind trial neither the physician nor the patient should know which intervention the patients get. They might get the group code or in drug trials only the code number for the blinded drug containers.

If online databases are used, a safer approach is combining two independent databases (Vickers 2006) to ensure concealment. One database is used to register the new patient. The information is then forwarded to a second database which is only accessed by an independent person and the information is then sent to the appropriate person to ensure the patient has the correct, randomized treatment. In small studies where setting up an online database or a central telephone hotline is too complex and expensive, sequentially numbered, opaque, sealed envelopes can be used. However, here it should be ensured that the envelopes really are opaque and that they are sealed so that cheating is impossible.

Randomization only ensures that patients in the groups are comparable for baseline characteristics, thus avoiding unknown confounders that may influence outcome, but will then be equally distributed to all the interventions in the study. This is vital and allows us to avoid bias. If patients were randomized to a group for which they had no preference they might be disappointed. This could influence how they rate the treatment outcome or result in more co-interventions. The same can occur in the provider rating; if they have special interest in one of the treatments their bias can influence the outcome. If a study does not use blinding it is classified as an open-label study, with all the disadvantages and bias that entails.

A single-blind trial is a study in which either the patient or the provider is unaware of the assignment of the patient. In reality it is mainly the patient who is blind. In a double-blind trial neither the patient nor the provider is aware of the patient’s treatment assignment. Sometimes trials are described as double-blind if patients and outcome assessors (not the provider) are blinded and this can be very misleading. Triple- and quadruple-blind trials are possible when, in addition to patients and providers, the outcome assessors and statisticians are blinded. To minimize potential biases treatment providers, patients and outcome assessors and even the statisticians should be blinded for the group allocation before the analysis is completed. This is possible in most pharmaceutical research; however if two clearly distinguishable treatments are compared patients and providers cannot be blinded. This occurs when comparing non-pharmacological intervention such as acupuncture or relaxation exercises with conventional treatment.

The issue of placebo is also much more complex and involved in CAM than it is in a simple drug trial; it is always more difficult to provide a real and convincing placebo which imitates the smell and the taste of a herbal medicine product or the visual and sensory effect of an acupuncture needle (Dube et al. 2007). However, even in CAM, when the verum and the placebo are indistinguishable, blinding might influence the providers’ treatment decisions. This is often discussed as a problem for trials of individualized homeopathy. In this type of homeopathy the provider prescribes an individual homeopathic remedy which should cover a whole range of symptoms for an individual patient (Witt et al. 2005). An option for the follow-up visits is usually to change the homeopathic remedy if no improvement is reported by the patient. This decision is based on an additional patient consultation, which can easily be influenced by the uncertainty of the provider (the homeopath) not knowing if there is no effect because the patient was allocated to the placebo group or whether the lack of response is due to the prescription of the wrong homeopathic remedy.

A vitally important factor for the success of blinding is to maintain it throughout the entire course of the trial. To assess if blinding is maintained throughout the study, patients, providers and assessors should be asked to guess the group allocation of each patient and the frequency of correct guesses can be compared between groups. However, if the verum is much more effective than the placebo, due to the effect of treatment, unblinding may happen. Asking these questions about blinding at an earlier stage does not solve the problem. We want to know if the patients were blinded at the time point when the primary outcome was assessed.

It is also helpful to assess the credibility of a treatment but this evaluates a slightly different dimension to blinding. Scales like this which have been used by Vincent (1990), are helpful and should be used in the early stages of the treatment.

In general there are four types of treatment that the patient can receive in the control group (Chow & Liu 2004): (1) placebo; (2) no treatment; (3) other active treatments; and (4) different dose or regime of the verum intervention.

The influence of patient expectations on outcomes is related to both within-group changes (from baseline to follow-up) and between-group differences (between the ‘real’ treatment and placebo). These suggestions are supported by two systematic reviews (Crow et al., 1999 and Mondloch et al., 2001).

If patients included in a trial have higher expectations of a positive outcome than the ‘average’ patient then this could result in within-group changes which are larger than in a more representative sample. High expectations might also be associated with high response rates and improved outcomes in the placebo control group. This could result in a ceiling effect, making it more difficult to detect a significant difference between verum and placebo if everyone improves because they expect and believe the treatment will work for them. By pooling data from four acupuncture trials we were able to show a significant association between greater improvement and high outcome expectations (Linde et al. 2007).

In order to account for patient expectations and to relate this to the main outcomes, patient expectations should be assessed at baseline before randomization. A reliable and valid, yet simple, tool for measuring aspects of expectations in clinical trials is still needed. Meanwhile simple questions such as: ‘How effective do you consider the treatment in general?’ with responses such as ‘very effective, effective, slightly effective, not effective, don’t know’ should be used routinely in trials involving the assessment of chronic pain and other chronic benign conditions where expectation and belief may be important predictors of treatment outcome.

Pragmatic studies aim to evaluate whether an intervention is effective under the sort of conditions that a patient might find in a clinic. These trials have model validity as far as the intervention is concerned. For instance, an acupuncture or herbal treatment may be adapted to suit some trial conditions in a very rigorous, placebo-controlled experiment and this may not really reflect good clinical practice. The resulting study may be of high scientific quality, but in effect is of little use to clinicians and health care providers, as one may not be able to generalize to the wider population from the results obtained. Pragmatic studies use wide inclusion criteria, allow flexibility in the treatment regimen and often focus on patient-centred outcomes such as quality of life or self-recorded pain (Zwarenstein 2008). Experimental or explanatory studies address whether an intervention works under controlled conditions. These studies typically use placebos or standardized interventions as controls. Experimental studies can have a high internal validity whereas pragmatic studies may have less internal validity but are more generalizable and relevant to clinical practice (see Chapter 1). The external validity of a study depends on the following aspects: (1) study participants; (2) interventions; and (3) outcome measures.

The generalizability of the study results depends on whether the group of study participants is representative of the usual type of patients with this specific condition. In experimental studies homogeneous patient groups are preferred, with the aim of reducing bias and enlarging internal validity. These patients are rarely representative of the people suffering from the disease under investigation, for example elderly patients over 70 years of age often suffer from cardiovascular diseases but because of their comorbidity they are often excluded from experimental studies on cardiovascular drugs. Within the context of health service research and decision-making it makes sense to investigate a treatment in a representative manner and as it is used in routine clinical care (Witt 2009). Therefore if you wished to evaluate whether acupuncture should be reimbursed by health insurance companies it would be better to assess the broad range of acupuncture styles likely to be found in practice instead of using a standardized treatment protocol.

This tension remains unresolved within the context of clinical trials in CAM and indeed conventional medicine; there are many studies in which CAM treatments have been applied in an inadequate manner and without model validity in order to improve rigour, reduce bias or increase feasibility. Such trials are very difficult to interpret and may be very of little value in the context of clinical practice and health care decision-making.

Using patient-centred outcomes should include taking relevant and valid measures that are appropriate to the condition being studied and ideally incorporating follow-up periods which are long enough to provide information about long-term effects. In pragmatic studies more subjective symptom-oriented outcomes such as symptom diaries or disease-specific quality-of-life questionnaires are often used. In contrast, more conventionally structured experimental studies may involve ‘hard’ outcome parameters such as laboratory measurements, if the condition within the study allows these measures to be usefully employed.

Most CAM trials in chronic benign diseases such as migraine and osteoarthritis have study observation periods of about 6 months at the most. Long-term follow-up in these trials is rare. This is unfortunate as health care providers would ideally like to be able to make thoughtful decisions that take into account the long-term effects of an intervention.

In reality there is often a compromise between internal and external validity in the design of any trial and this is usually and sensibly based on the study or research question. If the aim of the study is to give guidance to health service purchasers, a more population-based and ‘pragmatic’ approach is needed. If the aim is to develop a new intervention and determine its efficacy over placebo, an experimental study is more likely to produce the most valuable results. As mentioned in Chapter 1, CAM treatments are in general very diverse, widely used and widely available before they are evaluated scientifically. Consequently pragmatic studies can provide helpful information enabling medical decision-making. Ideally it would be best to have information from both experimental and pragmatic studies for all treatments, but this is not usually possible.

This chapter describes the main principles of RCT design, particularly with respect to CAM. RCTs were developed for the pharmaceutical industry, ideally to test the specific effects of drugs. They can usefully be adapted to evaluate a variety of different CAM techniques but this needs to be done thoughtfully and appropriately. The most important issue with respect to the design of an RCT is the research question. If you want to consider an RCT design, do you have a specific question that is best answered through an RCT? The advent of the RCT is both a blessing and a curse; it has certainly improved our understanding of medical evidence, but inevitably it is limited in its scope. Good science and clinical relevance do not always go hand in hand, and attempting to achieve both simultaneously requires clear thinking and the ability to work within a research team.