Boosting medical diagnostics by pooling independent judgments

Ralf H J M Kurvers, Stefan M. Herzog, Ralph Hertwig, Jens Krause, Patricia A. Carney, Andy Bogart, Giuseppe Argenziano, Iris Zalaudek, Max Wolf

Research output: Contribution to journalArticle

35 Citations (Scopus)

Abstract

Collective intelligence refers to the ability of groups to outperform individual decision makers when solving complex cognitive problems. Despite its potential to revolutionize decisionmaking in a wide range of domains, including medical, economic, and political decision making, at present, little is known about the conditions underlying collective intelligence in real-world contexts. We here focus on two key areas of medical diagnostics, breast and skin cancer detection. Using a simulation study that draws on large real-world datasets, involvingmore than 140 doctors makingmore than 20,000 diagnoses, we investigate when combining the independent judgments of multiple doctors outperforms the best doctor in a group. We find that similarity in diagnostic accuracy is a key condition for collective intelligence: Aggregating the independent judgments of doctors outperforms the best doctor in a group whenever the diagnostic accuracy of doctors is relatively similar, but not when doctors' diagnostic accuracy differs too much. This intriguingly simple result is highly robust and holds across different group sizes, performance levels of the best doctor, and collective intelligence rules. The enabling role of similarity, in turn, is explained by its systematic effects on the number of correct and incorrect decisions of the best doctor that are overruled by the collective. By identifying a key factor underlying collective intelligence in two important real-world contexts, our findings pave the way for innovative and more effective approaches to complex real-world decision making, and to the scientific analyses of those approaches.

Original languageEnglish
Pages (from-to)8777-8782
Number of pages6
JournalProceedings of the National Academy of Sciences of the United States of America
Volume113
Issue number31
DOIs
Publication statusPublished - Aug 2 2016

Fingerprint

Intelligence
Decision Making
Medical Economics
Aptitude
Skin Neoplasms
Breast Neoplasms

ASJC Scopus subject areas

  • General

Cite this

Kurvers, R. H. J. M., Herzog, S. M., Hertwig, R., Krause, J., Carney, P. A., Bogart, A., ... Wolf, M. (2016). Boosting medical diagnostics by pooling independent judgments. Proceedings of the National Academy of Sciences of the United States of America, 113(31), 8777-8782. https://doi.org/10.1073/pnas.1601827113

Boosting medical diagnostics by pooling independent judgments. / Kurvers, Ralf H J M; Herzog, Stefan M.; Hertwig, Ralph; Krause, Jens; Carney, Patricia A.; Bogart, Andy; Argenziano, Giuseppe; Zalaudek, Iris; Wolf, Max.

In: Proceedings of the National Academy of Sciences of the United States of America, Vol. 113, No. 31, 02.08.2016, p. 8777-8782.

Research output: Contribution to journalArticle

Kurvers, RHJM, Herzog, SM, Hertwig, R, Krause, J, Carney, PA, Bogart, A, Argenziano, G, Zalaudek, I & Wolf, M 2016, 'Boosting medical diagnostics by pooling independent judgments', Proceedings of the National Academy of Sciences of the United States of America, vol. 113, no. 31, pp. 8777-8782. https://doi.org/10.1073/pnas.1601827113
Kurvers, Ralf H J M ; Herzog, Stefan M. ; Hertwig, Ralph ; Krause, Jens ; Carney, Patricia A. ; Bogart, Andy ; Argenziano, Giuseppe ; Zalaudek, Iris ; Wolf, Max. / Boosting medical diagnostics by pooling independent judgments. In: Proceedings of the National Academy of Sciences of the United States of America. 2016 ; Vol. 113, No. 31. pp. 8777-8782.
@article{6692e8cc28ac4304af4a4ae281f113bc,
title = "Boosting medical diagnostics by pooling independent judgments",
abstract = "Collective intelligence refers to the ability of groups to outperform individual decision makers when solving complex cognitive problems. Despite its potential to revolutionize decisionmaking in a wide range of domains, including medical, economic, and political decision making, at present, little is known about the conditions underlying collective intelligence in real-world contexts. We here focus on two key areas of medical diagnostics, breast and skin cancer detection. Using a simulation study that draws on large real-world datasets, involvingmore than 140 doctors makingmore than 20,000 diagnoses, we investigate when combining the independent judgments of multiple doctors outperforms the best doctor in a group. We find that similarity in diagnostic accuracy is a key condition for collective intelligence: Aggregating the independent judgments of doctors outperforms the best doctor in a group whenever the diagnostic accuracy of doctors is relatively similar, but not when doctors' diagnostic accuracy differs too much. This intriguingly simple result is highly robust and holds across different group sizes, performance levels of the best doctor, and collective intelligence rules. The enabling role of similarity, in turn, is explained by its systematic effects on the number of correct and incorrect decisions of the best doctor that are overruled by the collective. By identifying a key factor underlying collective intelligence in two important real-world contexts, our findings pave the way for innovative and more effective approaches to complex real-world decision making, and to the scientific analyses of those approaches.",
author = "Kurvers, {Ralf H J M} and Herzog, {Stefan M.} and Ralph Hertwig and Jens Krause and Carney, {Patricia A.} and Andy Bogart and Giuseppe Argenziano and Iris Zalaudek and Max Wolf",
year = "2016",
month = "8",
day = "2",
doi = "10.1073/pnas.1601827113",
language = "English",
volume = "113",
pages = "8777--8782",
journal = "Proceedings of the National Academy of Sciences of the United States of America",
issn = "0027-8424",
number = "31",

}

TY - JOUR

T1 - Boosting medical diagnostics by pooling independent judgments

AU - Kurvers, Ralf H J M

AU - Herzog, Stefan M.

AU - Hertwig, Ralph

AU - Krause, Jens

AU - Carney, Patricia A.

AU - Bogart, Andy

AU - Argenziano, Giuseppe

AU - Zalaudek, Iris

AU - Wolf, Max

PY - 2016/8/2

Y1 - 2016/8/2

N2 - Collective intelligence refers to the ability of groups to outperform individual decision makers when solving complex cognitive problems. Despite its potential to revolutionize decisionmaking in a wide range of domains, including medical, economic, and political decision making, at present, little is known about the conditions underlying collective intelligence in real-world contexts. We here focus on two key areas of medical diagnostics, breast and skin cancer detection. Using a simulation study that draws on large real-world datasets, involvingmore than 140 doctors makingmore than 20,000 diagnoses, we investigate when combining the independent judgments of multiple doctors outperforms the best doctor in a group. We find that similarity in diagnostic accuracy is a key condition for collective intelligence: Aggregating the independent judgments of doctors outperforms the best doctor in a group whenever the diagnostic accuracy of doctors is relatively similar, but not when doctors' diagnostic accuracy differs too much. This intriguingly simple result is highly robust and holds across different group sizes, performance levels of the best doctor, and collective intelligence rules. The enabling role of similarity, in turn, is explained by its systematic effects on the number of correct and incorrect decisions of the best doctor that are overruled by the collective. By identifying a key factor underlying collective intelligence in two important real-world contexts, our findings pave the way for innovative and more effective approaches to complex real-world decision making, and to the scientific analyses of those approaches.

AB - Collective intelligence refers to the ability of groups to outperform individual decision makers when solving complex cognitive problems. Despite its potential to revolutionize decisionmaking in a wide range of domains, including medical, economic, and political decision making, at present, little is known about the conditions underlying collective intelligence in real-world contexts. We here focus on two key areas of medical diagnostics, breast and skin cancer detection. Using a simulation study that draws on large real-world datasets, involvingmore than 140 doctors makingmore than 20,000 diagnoses, we investigate when combining the independent judgments of multiple doctors outperforms the best doctor in a group. We find that similarity in diagnostic accuracy is a key condition for collective intelligence: Aggregating the independent judgments of doctors outperforms the best doctor in a group whenever the diagnostic accuracy of doctors is relatively similar, but not when doctors' diagnostic accuracy differs too much. This intriguingly simple result is highly robust and holds across different group sizes, performance levels of the best doctor, and collective intelligence rules. The enabling role of similarity, in turn, is explained by its systematic effects on the number of correct and incorrect decisions of the best doctor that are overruled by the collective. By identifying a key factor underlying collective intelligence in two important real-world contexts, our findings pave the way for innovative and more effective approaches to complex real-world decision making, and to the scientific analyses of those approaches.

UR - http://www.scopus.com/inward/record.url?scp=84982701411&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84982701411&partnerID=8YFLogxK

U2 - 10.1073/pnas.1601827113

DO - 10.1073/pnas.1601827113

M3 - Article

AN - SCOPUS:84982701411

VL - 113

SP - 8777

EP - 8782

JO - Proceedings of the National Academy of Sciences of the United States of America

JF - Proceedings of the National Academy of Sciences of the United States of America

SN - 0027-8424

IS - 31

ER -