A new calibration test and a reappraisal of the calibration belt for the assessment of prediction models based on dichotomous outcomes

Giovanni Nattino, Stefano Finazzi, Guido Bertolini

Research output: Contribution to journalArticle

Abstract

Calibration is one of the main properties that must be accomplished by any predictive model. Overcoming the limitations of many approaches developed so far, a study has recently proposed the calibration belt as a graphical tool to identify ranges of probability where a model based on dichotomous outcomes miscalibrates. In this new approach, the relation between the logits of the probability predicted by a model and of the event rates observed in a sample is represented by a polynomial function, whose coefficients are fitted and its degree is fixed by a series of likelihood-ratio tests. We propose here a test associated with the calibration belt and show how the algorithm to select the polynomial degree affects the distribution of the test statistic. We calculate its exact distribution and confirm its validity via a numerical simulation. Starting from this distribution, we finally reappraise the procedure to construct the calibration belt and illustrate an application in the medical context.

Original languageEnglish
Pages (from-to)2390-2407
Number of pages18
JournalStatistics in Medicine
Volume33
Issue number14
DOIs
Publication statusPublished - Jun 30 2014

Keywords

  • calibration test
  • dichotomous outcome models
  • goodness-of-fit
  • logistic regression models

ASJC Scopus subject areas

  • Medicine(all)

Fingerprint Dive into the research topics of 'A new calibration test and a reappraisal of the calibration belt for the assessment of prediction models based on dichotomous outcomes'. Together they form a unique fingerprint.

  • Cite this