We plan to discuss the principles and concepts that underpin randomized controlled trials (RCTs). This research methodology can be employed in a number of different contexts but RCTs are usually used in clinical settings. A trial, holding other factors constant, can be run to compare performance about a number of criteria, such as: ‘which treatment works better than a placebo?’ or ‘which device is most energy-efficient?’ The fundamental principles of RCTs remain the same in whichever context they are applied and this chapter aims to outline the steps that are important in setting up an RCT and meeting the requirements of a research protocol. RCTs allow us to answer very specific questions. They set out to evaluate the effect of a particular treatment or strategy in a population where an intervention is introduced, often by comparing the outcome with a control group where no intervention, or a standard intervention, may have been used. The population must be well defined and carefully selected.

A number of implicit assumptions underpin the RCT as follows (adapted from Vickers et al. 1997):

These tenets lead to the claim that the RCT provides the ’gold standard’ for research. It is the best means of attributing real cause and effect and therefore adds to our stock of knowledge. Not all researchers necessarily believe all of these, but they provide a sound framework for conducting a RCT.

In effect RCTs are mainly used to test a predefined hypothesis. When planning a confirmatory trial it is helpful to do a pilot trial with the aim of generating a hypothesis and to test the planned outcome measures and the study protocol for feasibility. In addition, pilot studies can be helpful in providing an idea of the effect size of the intervention and this in turn will inform the sample size for any larger, more definitive study. In these pilot trials statistics are used on an exploratory basis. The present published literature suggests that in CAM research confirmatory studies are often done without pilot trials and quite a large proportion of the confirmatory (definitive) studies seem like pilot studies and are too small to have enough power to detect a significant difference between groups. The reasons for this might be limited financial resources for CAM research and the fact that CAM has fewer qualified and experienced researchers than conventional medical research environments.

When planning a study it is important to clarify the study in more detail by considering the following aspects (Chow & Liu 2004):

Developing a hypothesis is an important issue when planning these studies. A hypothesis always consists of a null hypothesis (H₀) which assumes no effect and an alternative hypothesis (H_A) which holds the null hypothesis not to be true and consequently assumes an effect. The alternative hypothesis is more directly connected to the specific research question.

The aim of confirmatory studies is to answer research questions based on the hypothesis proposed by the researcher. Therefore, posing an adequate and answerable research question is essential as a clear, well-researched, thoughtful and specific question based on appropriate study design has a reasonable chance of providing a valid answer. One of the main reasons (other than failure to recruit) why the majority of research proposals are either unclear or fail to provide a useful answer is that the initial research question itself lacks clarity. The core elements of a precise research question for a confirmatory RCT can be summarized by PICO (patients, intervention, control intervention, outcome = primary endpoint; see example in Figure 5.1). Using PICO can be very helpful in developing your hypothesis.

The research question and the subsequent hypothesis build the basis for hypothesis testing. If the null hypothesis can be rejected based on a predefined significance level (e.g. 5% or 1%), the alternative hypothesis will be accepted. This means that, from the example shown in Figure 5.1, if there is a significant difference on the visual analogue scale (P < 0.05) between the acupuncture and the diclofenac group, the null hypothesis (H₀: acupuncture = diclofenac) can be rejected and the alternative hypothesis (H_A: acupuncture ≠ diclofenac) is applicable.

If there is no significant difference between both groups this does not necessarily mean that the null hypothesis is true as there may simply not be enough evidence to reject it. For example, if the sample size is too small the study may be underpowered. The chance of the study providing a statistically significant outcome is based on assuming that a 5% (or less) chance of this event occurring randomly is significant. This could mean that, with a P-value of 0.04 (a 4% chance of this happening randomly), the null hypothesis would be rejected, whereas with a P-value of 0.06 (a 6% chance of this happening randomly) there would not be enough evidence to reject the null hypothesis. The choice of a significance level of 5% is arbitrary but applicable throughout biological science and therefore the outcome of the study needs to be interpreted with caution, particularly with respect to the number of people entered. The greater sample size or number of people entered into the study, the more statistical ‘power’ it has and the more its statistical conclusions can be ‘trusted’.

However a significant difference between groups does not relate to the clinical importance of this finding as the statistical significance depends to a large extent on the sample size (number of people) and the variability of the condition within the study. Accordingly a large study might find a small but clinically unimportant difference between treatments, which is highly significant. Equally, a small study with few people might find a large difference that could be clinically important. Clinical importance describes a difference between two treatments that has a relevant effect size which is noticeable for, and valuable to, a patient with that condition.

Most randomized controlled studies evaluate if one intervention is superior to another. This is generally the case for treatment comparisons with a waiting list or a placebo intervention as controls. However, for some comparisons we test for similarity between two treatments (equivalence) or non- inferiority, but these equivalence studies, which include both non-inferiority and non-superiority trials, are rare in CAM. A non-inferiority hypothesis could, however, be very useful when comparing a CAM treatment with a conventional treatment. Non-inferiority or equivalence trials have methodological features that differ from superiority trials. In an equivalence trial, the null and alternative hypotheses are reversed. This means the null hypothesis is that treatment A is different from treatment B, whereas the alternative hypothesis is that there is no difference between the treatments.

In addition the effect size of the reference treatment has to be known and the margin for the non-inferiority has to be predefined for the power calculation and subsequent sample size. This margin represents the smallest value for a clinically relevant effect and outcomes within this margin are defined as non-inferior (Piaggio et al. 2006). The interpretation of the results depends on where the confidence interval lies relative to both the margin of non-inferiority and the null effect. For two-sided evidence (equivalence trials) two margins are needed (one above zero and one below) (Figure 5.2).

When planning a study probably the most important question is how many patients are needed to have enough statistical power to detect a clinically meaningful difference between the treatment groups. If the budget is limited or the medical conditions allow only a small number of patients to be included in the study, the study may fail to reach adequate power and could be of very limited value. An appropriate sample size is essential for a worthwhile study. The sample size calculation requires that the hypothesis should be clearly stated and a valid design and appropriate test statistics should be used. In hypothesis testing two errors can occur: the type I error (α) takes place when the null hypothesis is rejected although it was true and the type II error (β) is when the null hypothesis is not rejected when it was false. The probability of making a type I error is called the level of significance whereas the type II error is called power. A type I error is usually considered to be the most important of the two items this approach primarily tries to control for. To determine the sample size the investigator should provide the following information:

In clinical studies a significance level of 5% is usually chosen to reflect 95% confidence in the conclusions and usually a power of 80% or 90% is used in conjunction with this. Estimating a clinically meaningful difference and the standard deviation for the planned trial is often difficult, especially for CAM treatments, where there may be no previous data for the outcome for the intervention being evaluated. If the expected difference is large, fewer patients will be needed than if the expected difference is small. An important influence on the sample size is the variability of the patient outcomes for the primary outcome. If a high standard deviation is expected (lots of variability) more patients are needed to detect a significant difference between treatments than if the patient outcomes are more homogeneous.

Many CAM studies are conducted with small research budgets and consequently small sample sizes; therefore the sample size which would detect a significant difference between treatment groups may not be achieved. From a methodological perspective it is bad science to perform an underpowered study. If the budget is not sufficient to perform a trial of adequate size it is better to do no study, unless it is a pilot.

The RCT is the most reliable study design to detect whether the difference in the outcomes of two or more interventions is caused by a specific treatment. In effect it is the ‘gold standard’ for clinical research. With the aim for control for baseline differences, patients are allocated at random to receive one of several interventions. In addition, the treatment group is compared to at least one control group. Different control groups exist and the choice of control depends on the research question (Table 5.2). Placebo-controlled trials only tell us if the treatment is better than a placebo and we always need more information that that to inform clinical practice, where many treatments may be available for one condition.