Question

1. Which of the following best describes the concept of “heterogeneity of treatment effect” in clinical research?
A. Variation in how different subgroups of patients respond to the same treatment due to individual characteristics
B. Consistent treatment effects observed uniformly across all patient subgroups
C. Random measurement error affecting the treatment effect
D. Differences in treatment effect that arise only from study design flaws

2. In applying clinical trial results to patient care, expert judgment is primarily:
A. Used to provide a lower level of evidence in the absence of more authoritative trial data
B. Used to eliminate the need for randomized trials
C. Used to interpret and apply population-level trials to the individual patient
D. Useless, as it will eventually be completely replaced by trial data

3. A published case series describes 3 patients who developed a rare liver injury after taking drug X. Which statement about this study is most accurate?
A. It provides strong evidence that drug X causes liver injury.
B. It is equivalent in evidence quality to a randomized trial.
C. It can be used to accurately estimate the incidence of liver injury by drug X.
D. It is useful for generating hypotheses but cannot establish causality.

4. What is the primary strength of randomized controlled trials in clinical research?
A. They are always the fastest to get results.
B. They eliminate all forms of bias.
C. They minimize selection bias through randomization, thereby supporting causal inference.
D. If time and money are no barriers, they are always the best choice of study design.

5. What does a type 1 error represent in hypothesis testing?
A. Rejecting the null hypothesis when it is true
B. Failing to reject the null hypothesis when it is false
C. Concluding there is no effect when there actually is one
D. Finding statistical significance without clinical significance

6. If the pretest probability that a tested intervention works is very low (e.g., ~10%), what is the likelihood that a statistically significant finding (P <0.05) reflects a true positive?
A. It is high, because a statistically significant P value is a near-guarantee of the truth of the alternate hypothesis.
B. Pretest probability is irrelevant here, because α and β control error rates
C. It is low, because a low prior probability increases the chances of a false positive.
D. It is 95%, matching the confidence level.

7. A large clinical trial finds a difference between treatments with a P value of <0.001. Which of the following statements is most accurate?
A. The finding is highly significant and is therefore likely to be clinically important.
B. The low P value indicates the result will likely be replicated in another study.
C. A P value <0.001 ensures that the null hypothesis is false.
D. The finding is considered statistically significant, but there is not enough information to determine clinical significance.

8. Which of the following factors is most relevant when assessing the external validity of a clinical study?
A. The statistical power of the study
B. How closely the study population and conditions match the target population
C. The P value of the primary outcome
D. The quality of randomization

9. Critical appraisal tools are primarily used to:
A. Provide a checklist for authors on what to include in their manuscript
B. Guide clinicians through a structured evaluation of a study's validity and risk of bias
C. Guarantee the reliability of study results
D. Replace peer review by ensuring methodological quality

10. A trial observes no difference in its primary endpoint. A post-hoc analysis evaluating 10 secondary outcomes of interest identifies a statistically significant endpoint with P = 0.01. Which of the following is the most relevant statement regarding this finding?
A. This finding might be a false positive due to multiple comparisons.
B. This significant secondary outcome likely proves that the study drug is efficacious.
C. The trial should be disregarded entirely because the primary outcome was not met.
D. Post-hoc analyses are dubious and should be ignored.

Answer 1

1. A — Variation in how different subgroups of patients respond to the same treatment due to individual characteristics.

2. C — Used to interpret and apply population-level trials to the individual patient.

3. D — Useful for generating hypotheses but cannot establish causality.

4. C — They minimize selection bias through randomization, thereby supporting causal inference.

5. A — Rejecting the null hypothesis when it is true.

6. C — It is low, because a low prior probability increases the chances of a false positive.

7. D — The finding is statistically significant, but clinical significance cannot be determined from the P value alone.

8. B — How closely the study population and conditions match the target population.

9. B — Guide clinicians through a structured evaluation of a study's validity and risk of bias.

10. A — This finding might be a false positive due to multiple comparisons.