Predicting Outcome in the Intensive Care Unit Using Scoring Systems: Is New Better? A Comparison of SAPS and SAPS II in a Cohort of 1,393 Patients

OBJECTIVES. This study sought to compare the performance of the old and new versions of the Simplified Acute Pysiology Score, SAPS and SAPS II, in classifying patients according to the risk of hospital mortality. METHODS. To compare the performance of the two systems, measures of association between the scores and observed mortality were adopted, together with discrimination (area under the Receiver Operating Characteristics curve) and calibration (goodness-of-fit statistics) estimates. Subjects were 1,393 eligible patients recruited during 1 month in 1994. The outcome measure was vital status at hospital discharge. RESULTS. SAPS II was associated more strongly with hospital mortality than the earlier version. SAPS II also had better discrimination ability than SAPS (area under Receiver Operating Characteristics curve 0.80 versus 0.74) and predicted an overall number of deaths (416.5) closer to the observed figure (475) than SAPS II fitted our data. Both P values derived from goodness-of-fit statistics were lower than 0.05. CONCLUSIONS. SAPS II offers a real improvement compared with SAPS in its ability to explain hospital mortality, but its standard parameters do not fit our data from Italy. The role and impact of potential determinants of this lack of fit, such as random errors and confounders related to casemix and/or quality of care should be clarified before this scoring system be used outside formal research projects. Special caution is suggested when SAPS II is adopted to predict mortality to compare intensive care unit performance across different countries and systems of care.