Nonrandomized quality improvement intervention trials might overstate the strength of causal inference of their findings

Linda C. Li, Lorenzo Moja, Alberto Romero, Eric C. Sayre, Jeremy M. Grimshaw

Research output: Contribution to journalArticlepeer-review


Objective: To assess the strength of causal inferences reported in randomized and nonrandomized evaluations of quality improvement (QI) interventions in relation to the study design and the direction of results for the primary outcomes. Study Design and Setting: We searched 11 journals for QI intervention studies that aimed to change clinician behavior. Statements that addressed the causal inference between intervention and outcomes were extracted and were rated by 34 researchers for the strength of causality. Results: We found 38 randomized controlled trials (RCTs) and 35 non-RCTs, and extracted 68 quotes from the abstracts and 139 from the main text. A significant interaction was found between study design and direction of results for the abstract quotes (P = 0.022). The ratings for non-RCTs were higher when the results were mixed, but for RCTs, they were higher if the results were positive or no effect, although none of the differences were statistically significant at α = 0.05 after adjusting for multiple comparisons. For the main text quotes, the causality rating was higher by 0.43 for RCTs than for non-RCTs after adjusting for the direction of results (P <0.001). Conclusion: Authors might have overstated the strength of causal inference in the abstracts of non-RCTs, but appeared to report causality appropriately in the main text.

Original languageEnglish
Pages (from-to)959-966
Number of pages8
JournalJournal of Clinical Epidemiology
Issue number9
Publication statusPublished - Sep 2009


  • Causal inference
  • Causality
  • Quality improvement intervention
  • Research reporting
  • Review
  • Study design

ASJC Scopus subject areas

  • Epidemiology


Dive into the research topics of 'Nonrandomized quality improvement intervention trials might overstate the strength of causal inference of their findings'. Together they form a unique fingerprint.

Cite this