Validation of a novel radiographic scoring system for calcinosis affecting the hands of patients with systemic sclerosis

Objective. There are currently no validated outcome measures to assess calcinosis severity in systemic sclerosis (SSc; scleroderma). We sought to develop and validate a novel radiographic scoring system for calcinosis affecting the hands of SSc patients for potential use in future clinical trials. Methods. Following a 1-hour teleconference training session, 12 investigators (8 rheumatologists, 1 dermatologist, and 3 radiologists) scored 12 hand radiographs in random order using 2 scoring systems (termed "simple" and "complex") and re-scored 2 randomly assigned radiographs after a minimum of 24 hours. Interrater and intrarater reliability were assessed using a weighted kappa coefficient for the simple system (κ), and an intraclass correlation coefficient (ICC) for the complex system (ICC 0.7 for excellent). Results. Mean time to complete the complex scoring system was significantly longer than the simple scoring system (4.0 versus 0.4 minutes; P <0.0001). Overall interrater reliability for the simple scoring system was poor (κ = 0.39, 95% confidence interval [95% CI] 0.1-0.52) but improved if dichotomized as mild/moderate versus severe (κ = 0.51, 95% CI 0.26-0.7). Interrater reliability was excellent for the complex scoring system (ICC 0.89, 95% CI 0.86-0.92). Intrarater reliability was moderate for the simple scoring system (κ = 0.67, 95% CI 0.37-0.96) but almost perfect for the complex scoring system (ICC 0.93, 95% CI 0.89-0.97). Conclusion. We developed a novel radiographic scoring system that accounts for the area coverage, density, and anatomic location of calcinosis affecting the hands in patients with SSc. This scoring system is feasible with excellent reliability and should undergo further validation testing for use in clinical trials.