Research methodology – 2

Published on 23/05/2015 by admin

Filed under Psychiatry

Last modified 22/04/2025

Print this page

rate 1 star rate 2 star rate 3 star rate 4 star rate 5 star
Your rating: none, Average: 0 (0 votes)

This article have been viewed 1087 times

54 Research methodology – 2

Paul Wilkinson

  T F
1. Parametric statistics can be used if the distribution of results is Gaussian. image image
2. If standard deviations are larger, the effect size will be larger. image image
3. If p <0.05, there is a real difference between populations under study. image image
4. The standard error of the mean (SEM) is used in calculating confidence intervals for differences between means of continuous measures. image image
5. If multiple tests are performed without adjusting significance thresholds, then a type II error is more likely. image image
6. If a hypothesized difference between populations is small, fewer subjects are needed to make a study adequately powered. image image
7. Development of the randomized controlled trial means that other types of treatment studies with less rigorous methodology are obsolete and of no use. image image
8. Randomization ensures that treatment groups are similar on known and unknown variables. image image
9. Convergent validity is often measured by correlating overall scores on a new outcome scale with overall scores on an established scale. image image
10. The Ham-D is a good scale to use to diagnose depression. image image
11. Parametric tests use the difference in means and the standard deviation. image image
12. The Kruskal–Wallis Test is a non-parametric test that can be used to compare more than two unrelated groups. image image
13. The use of per protocol analysis is a good way to take into account differential dropout rates between treatment groups. image image
14. The use of intention-to-treat analysis is a way to deal with attrition bias. image image
15. The use of a placebo run-in may overestimate treatment effects. image image
16. In a case-control study, if an exposure of interest increases rates of hospital admission, there may be a Berkson bias. image image
17. In a case-control study of the association between the diagnosis of alcohol dependence and the diagnosis of depression, the relative risk is the statistic that would give the most useful information on strength of association. image image
18. If the lower boundary of the confidence interval for the odds ratio for lifetime cannabis use and schizophrenia is greater than 1, we can conclude that it is likely that there is a real association between cannabis use and schizophrenia. image image
19. In a case-control study, multiple linear regression can be used to prove that an exposure of interest causes an outcome of interest. image image
20. Multiple linear regression is an appropriate statistical technique to use if there is a curvilinear relationship between variables. image image
21. Before starting a screening programme, it is essential that there is a clear plan for what to do with cases screened as positive. image image
22. For a screening test for schizophrenia, positive predictive value will be higher for a random community sample than for a psychiatric inpatient sample. image image
23. In meta-analysis, publication bias may lead to a type I error. image image
24. For prevalence, a time period must always be given. image image
25. It is important to form an a priori hypothesis in qualitative research. image image

ANSWERS

1. Parametric statistics can be used if the distribution of results is Gaussian.

True: Parametric statistics, such as t-tests and ANOVAs, need normally distributed, or ‘Gaussian’, data.

2. If standard deviations are larger, the effect size will be larger.

False: If there is a wider spread of results, the distributions of the two samples will overlap more, and therefore there will be less real difference between samples. The difference between samples is often given as the ‘effect size’, or ‘Hedges’ g’, and is calculated by the difference in means divided by the weighted average standard deviation.

3. If p <0.05, there is a real difference between populations under study.

False: This just means that there is a high probability (>95%) that they are truly different.

4. The standard error of the mean (SEM) is used in calculating confidence intervals for differences between means of continuous measures.

True: The confidence interval is calculated with multiples of the difference in mean divided by the SEM for the difference. The SEM of a sample is the standard deviation divided by the square root of the sample size. It is therefore smaller as your sample gets bigger, and the confidence interval therefore becomes smaller as your sample sizes increase. When interpreting graphs with error lines, be sure to check whether standard deviation or SEM is used, as the SEM will appear much more impressive!

5. If multiple tests are performed without adjusting significance thresholds, then a type II error is more likely.

False: Type II errors occur when there is a real difference between populations, but a study finds no statistically significant difference. If a lot of statistical tests are performed on a dataset without adjusting significance thresholds, we are more likely to find a chance ‘significant’ result, when there is no real difference (i.e. a type I error). For example, if we perform 20 significance tests on samples where there is no real difference, the chances are that one of them (5%) will have a p value less than 5%. However, if multiple tests are performed without adjusting significance thresholds, there will be no effect on the chances of a type II error. If we reduce the significance thresholds, as we should to reduce type I errors, we do increase the rate of type II errors. Type II errors are generally less serious than type I errors. It is best to have a very small number of a priori hypotheses, to minimize chances of both types of errors.

6. If a hypothesized difference between populations is small, fewer subjects are needed to make a study adequately powered.

False: Before commencing a study, it is imperative to calculate the sample size needed to make a study adequately powered to find a statistically significant difference, if there is one. Power is increased (and a smaller sample size is needed) if: there is a larger hypothesized difference between groups; there is a smaller hypothesized standard deviation; parametric statistics can be used or if the necessary p value threshold is higher.

7. Development of the randomized controlled trial means that other types of treatment studies with less rigorous methodology are obsolete and of no use.

False: An RCT is the best study design, with the least bias; but other designs are useful if no RCTs have been performed. Other designs such as uncontrolled case series or open-label controlled trials are normally carried out to show there is a reasonable chance a treatment is effective before it is ethical to put patients in an RCT.

8. Randomization ensures that treatment groups are similar on known and unknown variables.

False: A major advantage of randomization is that it increases the chances that treatment groups are similar on known and unknown variables. If we non-randomly allocate to groups, we can ensure balance on known confounding variables, but there may still be an imbalance on important confounding variables we have not thought of. However, random allocation only increases the chance of balance. There may still be imbalance (‘randomization error’), especially if groups are small.

9. Convergent validity is often measured by correlating overall scores on a new outcome scale with overall scores on an established scale.

False: That is concurrent validity. Convergent validity is whether measures which are expected to be correlated are indeed associated, e.g. correlation of emotional, cognitive and physical symptoms on a depression scale.

10. The Ham-D is a good scale to use to diagnose depression.

False: The Ham-D is a continuous scale that gives the level of symptoms, not the diagnosis. As with all continuous scales, some people with a diagnosis will get lower scores than some people without a diagnosis. Hence, beware of studies that state that all subjects have a diagnosis because they score above cut-off on a rating scale. Participants should only be interpreted as having a diagnosis if they meet certain diagnostic criteria, e.g. DSM, ICD, preferably rated using a standardized and valid interview, e.g. SCID, SADS.

11. Parametric tests use the difference in means and the standard deviation.

True: As we know the distribution is normal, the means and standard deviations (together with the numbers of participants) give us enough information about the data to compare different samples.

12. The Kruskal–Wallis Test is a non-parametric test that can be used to compare more than two unrelated groups.

True: It is rather like the ANOVA, a test to compare more than two groups, which can be used only if data is normally distributed.

13. The use of per protocol analysis is a good way to take into account differential dropout rates between treatment groups.

False: In most randomized controlled trials, some participants will drop out of treatment, or change treatment group. We then need to decide which group these people who have changed group should be analysed in: it could be as treated (what treatment they got); per protocol (only those who received the allocated treatment) or intention to treat (all participants analysed in the groups they were initially allocated to, ignoring the fact that they changed treatment). We should always use intention to treat (ITT) to take account of dropouts. Any other approach introduces bias. Also, ITT is more clinically useful: we need to know which treatment we should offer to our patients, when we do not know if they will drop out of treatment.

14. The use of intention-to-treat analysis is a way to deal with attrition bias.

True: Attrition bias is bias caused by differential dropout rates in different groups.

15. The use of a placebo run-in may overestimate treatment effects.

True: A placebo run-in is giving all participants a placebo and then including only those who do fail to respond to placebo in the main RCT. This increases any treatment effects. In real life, we are interested in the actual difference between a treatment and no treatment in the whole population; not the subpopulation of those who fail to respond quickly to placebo.

16. In a case-control study, if an exposure of interest increases rates of hospital admission, there may be a Berkson bias.

True: In the Berkson bias (admission rate bias or paradox), the exposure of interest leads to differential rates of hospital admission of cases and controls, rather than leading to different rates of the outcome of interest. For example, we may be more likely to admit a patient with depression if they have poor social support. A disproportionately high number of depressed inpatients would therefore have poor social support. A study where depressed cases are hospital inpatients may then make us think that there is an association between poor social support and depression. The solution is to try to use outpatient or community samples.

17. In a case-control study of the association between the diagnosis of alcohol dependence and the diagnosis of depression, the relative risk is the statistic that would give the most useful information on strength of association.

False: Unlike in cohort studies, we cannot calculate the relative risk from case-control studies, as we do not know the true numbers of people exposed to the exposure of interest. We must instead use the odds ratio, which approximates the relative risk if there is a very rare outcome.

18. If the lower boundary of the confidence interval for the odds ratio for lifetime cannabis use and schizophrenia is greater than 1, we can conclude that it is likely that there is a real association between cannabis use and schizophrenia.

True: An odds ratio (or relative risk) of 1 means there is no association at all between factors. If the CI includes 1, this shows that the range of likely associations includes 1, i.e. no real difference. If all points of the confidence interval are greater than 1, we can conclude that it is likely there is a real association.

19. In a case-control study, multiple linear regression can be used to prove that an exposure of interest causes an outcome of interest.

False: Observational studies show only a likely association between exposures and outcomes of interest. For example, the outcome measure may cause the exposure, or a third, confounding, factor, may cause both the exposure and outcome of interest. Randomized studies are needed to show ‘causation’.

20. Multiple linear regression is an appropriate statistical technique to use if there is a curvilinear relationship between variables.

False: There must be linear relationship.

21. Before starting a screening programme, it is essential that there is a clear plan for what to do with cases screened as positive.

True: Screening is expensive and potentially harmful (especially by causing unnecessary anxiety in people with false-positive screens) and needs to be thought through. There must be a clear plan for what to do with people who screen positive, so that the benefits of early diagnosis outweigh the unnecessary anxiety.

22. For a screening test for schizophrenia, positive predictive value will be higher for a random community sample than for a psychiatric inpatient sample.

False: The positive predictive value of the test is the proportion of those screened positive that truly have the diagnosis. If there is lower prevalence, a higher proportion of the sample will not have the diagnosis and a higher proportion of those screened positive similarly will not have the diagnosis.

23. In meta-analysis, publication bias may lead to a type I error.

True: There is often a bias in favour of publishing positive findings. If only these studies are included, treatment effect is inflated. This may lead to the wrong conclusion that a treatment is better than a comparator.

24. For prevalence, a time period must always be given.

True: We need to say if it is point prevalence (at one point in time) or period prevalence (diagnosis at any time point during a time period).

25. It is important to form an a priori hypothesis in qualitative research.

False: In quantitative research we should form a hypothesis in advance (a priori). We then collect numerical data and use statistical tests to see how likely our hypothesis is to be correct. We can also test hypotheses in qualitative research though it is also often used to form hypotheses. Researchers try not to have pre-formed ideas and through methods such as focus groups and in-depth interviews, ideas and themes emerge. These can then be formulated as hypotheses to be tested in future research.

Related posts:

Antidepressants Learning disability – 5 Statistics – 4 Psychotherapy – 2 Old age psychiatry – 1 Antipsychotics