Research methodology – 1

Published on 23/05/2015 by admin

Filed under Psychiatry

Last modified 22/04/2025

Print this page

This article have been viewed 1049 times

53 Research methodology – 1

	T	F
1. For a skewed distribution, the median is a more useful measure of central tendency than the mean.
2. When comparing recovery rates between two samples, parametric statistics are best.
3. For two studies with identical effect sizes, the p value will be smaller for the study with much larger numbers.
4. If the confidence interval for the difference in Ham-D scores between treatment groups includes 0, we can conclude that it is likely that there is a real difference between treatments that is not due to chance.
5. A type I error means that a result is falsely found to be statistically significant.
6. A very low p value indicates that a result is clinically significant.
7. An effectiveness study for a treatment for depression is more likely to include patients with co-morbid conditions than an efficacy study.
8. The need for informed consent can limit the external validity of treatment studies.
9. A scale must have good reliability for it to have good validity.
10. We should assess the criterion validity of new scales that measure depressive symptoms.
11. Log transformation is sometimes essential before parametric statistical tests are used.
12. Non-parametric tests use the difference in medians and the inter-quartile range.
13. Not taking into account dropouts from treatment may introduce bias.
14. The use of intention to treat analysis may reduce accuracy of results.
15. Subjects in cluster randomized controlled trials (RCTs) are analysed in exactly the same way as in standard RCTs.
16. Crossover studies can have more power than using different patients in different treatment groups.
17. Stratification can be used to control for confounding in a case-control study.
18. The t-test is commonly used to look at strength of association in case-control studies with a dichotomous exposure of interest.
19. Multiple logistic regression is used to investigate the effects of variables on a continuous outcome measure.
20. In multiple linear regression, if two covariates are highly correlated, there may be a type II error.
21. Multiple linear regression cannot be used if there is a heavily skewed distribution of a proposed variable.
22. For a screening test for schizophrenia, sensitivity will probably be lower for a random community sample than for a psychiatric inpatient sample.
23. A meta-analysis will give more accurate and less biased results than one RCT.
24. If the duration of illness is very long, incidence will be higher than prevalence.
25. Administering the Beck Depression Inventory to a population is an accurate method of ascertaining population prevalence of depression.

ANSWERS

1. For a skewed distribution, the median is a more useful measure of central tendency than the mean.

True: The median gives the middle value, i.e. the 50th highest value if there are 99 values. In a skewed distribution, more values are a long way from the average, to one side of it. If positively skewed, values above the median are further from the median than values below the median. The mean will be over-affected by outliers if the distribution is skewed.

2. When comparing recovery rates between two samples, parametric statistics are best.

False: When looking at recovery rates you are looking at numbers, or counts, of participants in different groups rather than comparing two outcome curves. Therefore, parametric statistics, which are used to compare two or more normal curves, cannot be used. Often the chi-squared test is used.

3. For two studies with identical effect sizes, the p value will be smaller for the study with much larger numbers.

True: If there are larger samples we have more confidence that the difference is not due to chance. The statistics we calculate, for example t or F values, are often proportional to the square root of the number of participants.

4. If the confidence interval for the difference in Ham-D scores between treatment groups includes 0, we can conclude that it is likely that there is a real difference between treatments that is not due to chance.

False: If the confidence interval includes 0, this shows that the range of likely differences between populations includes 0, i.e. there is no actual difference.

5. A type I error means that a result is falsely found to be statistically significant.

True: In other words, the p value is below your threshold (often 5%) but there is not really a difference between two populations (such as all depressed people given a new antidepressant and all depressed people given a placebo). The difference is just a chance finding in a study. This can be important as we may conclude from a study that a new treatment is better than placebo when in fact there is no real difference. Bias, where poor methodology causes differences between groups, increases the risk of type I errors.

6. A very low p value indicates that a result is clinically significant.

False: A very low p value just means the difference between samples is very unlikely to be due to chance (statistical significance). It does not mean that the results are clinically meaningful. We must use our clinical judgement to decide that.

7. An effectiveness study for a treatment for depression is more likely to include patients with co-morbid conditions than an efficacy study.

True: Effectiveness studies are more likely to mirror real life clinical practice, such as including all possible patients with a condition, including those with co-morbid conditions. The results of these studies are therefore generally more relevant to clinical practice than efficacy studies, whose aim is to show whether a treatment can work in ideal circumstances.

8. The need for informed consent can limit the external validity of treatment studies.

True: External validity refers to how much the results of a study are generalizable to the population of interest. Unless there are exceptional circumstances, we can recruit only consenting subjects to research studies. If a lot of a population under study would not have the capacity to consent, such as patients suffering from psychosis, a study sample would not be fully representative of the population we want to treat. Research ethics committees have a difficult job in balancing the rights of the individual against the need for best information on treating a population when deciding whether there are ‘exceptional circumstances’ to allow inclusion of participants who do not have the capacity to give consent.

9. A scale must have good reliability for it to have good validity.

True: Validity (more strictly, internal validity) is how well a scale measures what it is supposed to measure. Reliability is whether the same scale gives the same results with the same subject when measured at different times by the same rater (intra-rater reliability) or different raters (inter-rater reliability). If a scale is not reliable, its measurements will not be valid. On the other hand, validity is not essential for reliability.

10. We should assess the criterion validity of new scales that measure depressive symptoms.

False: Criterion validity is how well a scale compares with an existing known quantity, such as height, or psychiatric diagnosis. It can be useful to compare a scale against DSM or ICD diagnosis, but it is not essential. Scales are more important in measuring the severity of symptoms and the changes in symptoms. It is more useful to test their reliability and concurrent validity, i.e. comparison against other valid rating scales.

11. Log transformation is sometimes essential before parametric statistical tests are used.

True: We need normally-distributed results to use parametric statistics. If results are skewed, we can log transform all values (in all groups!), and if the distributions are then normal, we can use parametric statistics.

12. Non-parametric tests use the difference in medians and the inter-quartile range.

False: Descriptive statistics like medians and inter-quartile ranges do not give us enough information to compare non-normally distributed groups. Instead, we must look at outcome scores of all participants in all groups and rank them from highest to lowest. We can then look at the total ranks of each group. Statistical tests, such as the Mann-Whitney test (for two independent samples) will tell us whether the difference in total ranks between the groups is statistically significant.

13. Not taking into account dropouts from treatment may introduce bias.

True: The reasons for dropout are likely to differ between groups. If one considers a study of psychotherapy for depression vs. waiting list control, it may be that the most unwell people in the treatment group are unable to travel to psychotherapy sessions. They may be more likely to drop out of the study than the equally unwell people on the waiting list. This would introduce systematic non-random error which would inflate the apparent efficacy of psychotherapy.

14. The use of intention to treat analysis may reduce accuracy of results.

True: You are not measuring outcome based on treatment given, so some accuracy may be lost. However, ITT reduces the effects of unknown bias caused by changes in treatment and is essential, as this bias cannot be measured. ITT may increase or decrease accuracy (we never know!) but it does minimize the chance of biased results.

15. Subjects in cluster randomized controlled trials (RCTs) are analysed in exactly the same way as in standard RCTs.

False: Cluster trials are RCTs where groups (for example GP practices or psychiatric units) rather then individual patients are randomly allocated to treatments. When analysing cluster RCTs we need to take into account intra-cluster correlation – participants in a cluster (such as GP practices) are likely to have certain things in common that they do not share with participants in other clusters. This may be because of shared environmental factors or different socio-economic status in different catchment areas. Because of these shared characteristics there may be large pre-treatment differences in outcome or risk variables in different groups. We can estimate the intraclass correlation and cluster size and use these to calculate a ‘design effect’. We then increase the variance term in our calculations by this design effect.

16. Crossover studies can have more power than using different patients in different treatment groups.

True: In crossover studies, participants act as their own controls, alternating between different treatments at different time points. We therefore do not have to take into account possible error caused by between-subjects differences. Instead, more powerful repeat measures statistics can be used.

17. Stratification can be used to control for confounding in a case-control study.

True: Results are stratified into levels of a confounding factor, e.g. high levels, low levels or no levels of past cannabis use. We can look for an association between exposure of interest and outcome of interest at all strata, or use the Mantel–Haenszel procedure to calculate the association, controlling for this confounder. Stratification can also be used in cohort studies.

18. The t-test is commonly used to look at strength of association in case-control studies with a dichotomous exposure of interest.

False: We instead use the confidence interval of the odds ratio or the chi squared test. The t-test is used to compare groups with continuously distributed exposure of interest.

19. Multiple logistic regression is used to investigate the effects of variables on a continuous outcome measure.

False: Multiple logistic regression is used to investigate the effects of variables on a binary outcome measure, e.g. diagnosis/no diagnosis. Multiple linear regression is used to investigate the effects of variables on a continuous outcome measure.

20. In multiple linear regression, if two covariates are highly correlated, there may be a type II error.

True: In multiple regression, if two covariates correlate very highly (e.g. 0.9), the one with minutely better correlation with the outcome measure is selected first. The other variable will supposedly be a much less accurate predictor. This is called multicollinearity. This can lead to type II errors, as exposures of interest are wrongly deemed not to be associated with the outcome of interest.

21. Multiple linear regression cannot be used if there is a heavily skewed distribution of a proposed variable.

False: Normally-distributed data are needed. However, we can transform the distribution (e.g. log transformation) to make it normal.

22. For a screening test for schizophrenia, sensitivity will probably be lower for a random community sample than for a psychiatric inpatient sample.

True: Sensitivity is the proportion of those who truly have the diagnosis who are positive in a screening test. Sensitivity does not vary with prevalence. However, a lower severity of illness in a community group will increase the numbers who score below cut-off, thus reducing sensitivity.

23. A meta-analysis will give more accurate and less biased results than one RCT.

False: Not always. A good RCT is better than a meta-analysis of poor studies.

24. If the duration of illness is very long, incidence will be higher than prevalence.

False: Incidence is the proportion of a population with a new onset of illness over a time period. Prevalence is the proportion of a population who have an illness at any point during a time period. If there is long ongoing illness, relatively few get a new diagnosis and so incidence is lower than prevalence.

25. Administering the Beck Depression Inventory to a population is an accurate method of ascertaining population prevalence of depression.

False: The BDI is a self-rated measure of severity of depressive symptoms. It does not give a diagnosis. It may, however, be used in the first stage of a prevalence study.

All respondents with a BDI above a cut-off can receive a full diagnostic interview and a smaller proportion of those with lower BDIs are also seen, to ascertain the depression prevalence in this larger group.

Related

1500 Questions in Psychiatry For the MRCPsych