A group of researchers conducted a prospective cohort study to evaluate the association between caffeine consumption and the risk of colon cancer in women. A total of 200 women age 40-60 participated in the study. The statistical analysis controlled for factors such as amount of caffeine consumed per day, caffeine consumption changes during follow-up time, and other factors related to both caffeine consumption and colon cancer. Results showed that caffeine consumption was associated with a statistically significant decrease in the risk of colon cancer with a relative risk of 0.83 (p = 0.02). Researchers concluded that women in general could significantly reduce the risk of colon cancer if they increased their caffeine consumption. Which of the following is most likely to invalidate the researchers' conclusion?
Internal validity | External validity | |
Characteristics | Describes causality (ie, if change in independent variable causes change in dependent variable) | Describes generalizability (ie, if observed relationship applies to situations or people outside study) |
↑ As study becomes more tightly controlled | ↓ As study becomes more tightly controlled | |
↓ As study becomes more like the real world | ↑ As study becomes more like the real world | |
Threats to validity | Bias due to:
| Bias due to:
|
The external validity (or generalizability) of a study is the applicability of study results beyond the specific population studied. This cohort study evaluated the association between caffeine consumption and the risk of colon cancer in women age 40-60 (ie, middle-aged women). Although study results showed that caffeine consumption was associated with a statistically significant decrease in the risk of colon cancer (ie, relative risk = 0.83; p = 0.02), it is unknown whether the association holds for other populations of women (eg, younger women age <40, elderly women age >60). Therefore, these study results may not apply to women in general.
(Choice B) Internal validity is concerned with how well results provide evidence of a causal relationship in a study; it is affected by bias (eg, confounding bias, selection bias). In this study, the statistical analysis controlled for potential confounders (ie, amount of caffeine consumed per day, caffeine consumption changes during follow-up time, other factors related to colon cancer) but no other description of how the study was designed or conducted was provided. Therefore, it is not possible to determine whether the internal validity was affected (ie, if bias is present).
(Choices C and D) Reproducibility (reliability) refers to the extent to which a measure produces consistent results if the measurements are repeated multiple times. Information on how repeated measurements of variables (eg, caffeine consumption, changes in consumption) were collected over time is not provided, so it is not possible to determine whether the measurements were reproducible (reliable).
(Choice E) Statistical power is the ability of a study to detect a significant association (eg, decrease in risk of colon cancer) when that association exists. This study was able to detect a statistically significant association between caffeine consumption and colon cancer, so the study had sufficient statistical power.
Educational objective:
The external validity (or generalizability) of a study is the applicability of results beyond the specific population studied.