Inter-observer variability of clinical target volume delineation in radiotherapy treatment of pancreatic cancer: A multi-institutional contouring experience

Luciana Caravatta, Gabriella Macchia, Gian C. Mattiucci, Aldo Sainato, Nunzia L V Cernusco, Giovanna Mantello, Monica Di Tommaso, Marianna Trignani, Antonino De Paoli, Gianni Boz, Maria L. Friso, Vincenzo Fusco, Marta Di Nicola, Alessio G. Morganti, Domenico Genovesi

Research output: Contribution to journalArticle

25 Citations (Scopus)

Abstract

Background: An observational multi-institutional study has been conducted aimed to evaluate the inter-observer variability in clinical target volume (CTV) delineation among different radiation oncologists in radiotherapy treatment of pancreatic cancer.Methods: A multi-institutional contouring dummy-run of two different cases of pancreatic cancer treated by postoperative and preoperative radiotherapy (RT) was performed. Clinical history, diagnostics, and planning CT imaging were available on AIRO website (http://www.radioterapiaitalia.it). Participants were requested to delineate CTVs according to their skills and knowledge. Aiming to quantify interobserver variability of CTVs delineations, the total volume, craniocaudal, laterolateral, and anteroposterior diameters were calculated. Descriptive statistic was calculated. The 95% Confidence Interval (95% CI) for coefficient of variation (CV) was estimated. The Dice Similarity Index (DSI) was used to evaluate the spatial overlap accuracy of the different CTVs compared with the CTVs of a national reference Centre considered as a benchmark. The mean DSI (mDSI) was calculated and reported.Results: A total of 18 radiation oncologists from different Institutes submitted the targets. Less variability was observed for the Elective CTV rather than the Boost CTV, in both cases. The estimated CV were 28.8% (95% CI: 21.2 - 45.0%) and 20.0% (95% CI: 14.9 - 30.6%) for the Elective CTV, in adjuvant (Case 1) and neoadjuvant (Case 2) case, respectively. The mDSI value was 0.68 for the Elective CTVs in both cases (range 0.19 - 0.79 in postoperative vs range 0.35 - 0.79 in preoperative case). The mDSI was increased to 0.71 (Case 1) and 0.72 (Case 2) if the observers with a worse agreement have been excluded. On the other hand, a CV of 42.4% (95% CI: 30.1 - 72.4%) and 63.8% (95% CI: 43.9 - 119.2%) with a mDSI value of 0.44 and 0.52, were calculated for the Boost CTV in Case 1 and Case 2, respectively.Conclusions: The CV and mDSI obtained values for Elective CTVs showed an acceptable agreement among participants either in postoperative as well in preoperative setting. Additional strategies to reduce the variability in Boost CTV delineation need to be found and promoted.

Original languageEnglish
Article number198
JournalRadiation Oncology
Volume9
Issue number1
DOIs
Publication statusPublished - Aug 9 2014

Fingerprint

Observer Variation
Pancreatic Neoplasms
Radiotherapy
Confidence Intervals
Therapeutics
Benchmarking

ASJC Scopus subject areas

  • Oncology
  • Radiology Nuclear Medicine and imaging

Cite this

Inter-observer variability of clinical target volume delineation in radiotherapy treatment of pancreatic cancer : A multi-institutional contouring experience. / Caravatta, Luciana; Macchia, Gabriella; Mattiucci, Gian C.; Sainato, Aldo; Cernusco, Nunzia L V; Mantello, Giovanna; Di Tommaso, Monica; Trignani, Marianna; De Paoli, Antonino; Boz, Gianni; Friso, Maria L.; Fusco, Vincenzo; Di Nicola, Marta; Morganti, Alessio G.; Genovesi, Domenico.

In: Radiation Oncology, Vol. 9, No. 1, 198, 09.08.2014.

Research output: Contribution to journalArticle

Caravatta, L, Macchia, G, Mattiucci, GC, Sainato, A, Cernusco, NLV, Mantello, G, Di Tommaso, M, Trignani, M, De Paoli, A, Boz, G, Friso, ML, Fusco, V, Di Nicola, M, Morganti, AG & Genovesi, D 2014, 'Inter-observer variability of clinical target volume delineation in radiotherapy treatment of pancreatic cancer: A multi-institutional contouring experience', Radiation Oncology, vol. 9, no. 1, 198. https://doi.org/10.1186/1748-717X-9-198
Caravatta, Luciana ; Macchia, Gabriella ; Mattiucci, Gian C. ; Sainato, Aldo ; Cernusco, Nunzia L V ; Mantello, Giovanna ; Di Tommaso, Monica ; Trignani, Marianna ; De Paoli, Antonino ; Boz, Gianni ; Friso, Maria L. ; Fusco, Vincenzo ; Di Nicola, Marta ; Morganti, Alessio G. ; Genovesi, Domenico. / Inter-observer variability of clinical target volume delineation in radiotherapy treatment of pancreatic cancer : A multi-institutional contouring experience. In: Radiation Oncology. 2014 ; Vol. 9, No. 1.
@article{08270bd38c8048d1a67bf2d60663984d,
title = "Inter-observer variability of clinical target volume delineation in radiotherapy treatment of pancreatic cancer: A multi-institutional contouring experience",
abstract = "Background: An observational multi-institutional study has been conducted aimed to evaluate the inter-observer variability in clinical target volume (CTV) delineation among different radiation oncologists in radiotherapy treatment of pancreatic cancer.Methods: A multi-institutional contouring dummy-run of two different cases of pancreatic cancer treated by postoperative and preoperative radiotherapy (RT) was performed. Clinical history, diagnostics, and planning CT imaging were available on AIRO website (http://www.radioterapiaitalia.it). Participants were requested to delineate CTVs according to their skills and knowledge. Aiming to quantify interobserver variability of CTVs delineations, the total volume, craniocaudal, laterolateral, and anteroposterior diameters were calculated. Descriptive statistic was calculated. The 95{\%} Confidence Interval (95{\%} CI) for coefficient of variation (CV) was estimated. The Dice Similarity Index (DSI) was used to evaluate the spatial overlap accuracy of the different CTVs compared with the CTVs of a national reference Centre considered as a benchmark. The mean DSI (mDSI) was calculated and reported.Results: A total of 18 radiation oncologists from different Institutes submitted the targets. Less variability was observed for the Elective CTV rather than the Boost CTV, in both cases. The estimated CV were 28.8{\%} (95{\%} CI: 21.2 - 45.0{\%}) and 20.0{\%} (95{\%} CI: 14.9 - 30.6{\%}) for the Elective CTV, in adjuvant (Case 1) and neoadjuvant (Case 2) case, respectively. The mDSI value was 0.68 for the Elective CTVs in both cases (range 0.19 - 0.79 in postoperative vs range 0.35 - 0.79 in preoperative case). The mDSI was increased to 0.71 (Case 1) and 0.72 (Case 2) if the observers with a worse agreement have been excluded. On the other hand, a CV of 42.4{\%} (95{\%} CI: 30.1 - 72.4{\%}) and 63.8{\%} (95{\%} CI: 43.9 - 119.2{\%}) with a mDSI value of 0.44 and 0.52, were calculated for the Boost CTV in Case 1 and Case 2, respectively.Conclusions: The CV and mDSI obtained values for Elective CTVs showed an acceptable agreement among participants either in postoperative as well in preoperative setting. Additional strategies to reduce the variability in Boost CTV delineation need to be found and promoted.",
author = "Luciana Caravatta and Gabriella Macchia and Mattiucci, {Gian C.} and Aldo Sainato and Cernusco, {Nunzia L V} and Giovanna Mantello and {Di Tommaso}, Monica and Marianna Trignani and {De Paoli}, Antonino and Gianni Boz and Friso, {Maria L.} and Vincenzo Fusco and {Di Nicola}, Marta and Morganti, {Alessio G.} and Domenico Genovesi",
year = "2014",
month = "8",
day = "9",
doi = "10.1186/1748-717X-9-198",
language = "English",
volume = "9",
journal = "Radiation Oncology",
issn = "1748-717X",
publisher = "BioMed Central",
number = "1",

}

TY - JOUR

T1 - Inter-observer variability of clinical target volume delineation in radiotherapy treatment of pancreatic cancer

T2 - A multi-institutional contouring experience

AU - Caravatta, Luciana

AU - Macchia, Gabriella

AU - Mattiucci, Gian C.

AU - Sainato, Aldo

AU - Cernusco, Nunzia L V

AU - Mantello, Giovanna

AU - Di Tommaso, Monica

AU - Trignani, Marianna

AU - De Paoli, Antonino

AU - Boz, Gianni

AU - Friso, Maria L.

AU - Fusco, Vincenzo

AU - Di Nicola, Marta

AU - Morganti, Alessio G.

AU - Genovesi, Domenico

PY - 2014/8/9

Y1 - 2014/8/9

N2 - Background: An observational multi-institutional study has been conducted aimed to evaluate the inter-observer variability in clinical target volume (CTV) delineation among different radiation oncologists in radiotherapy treatment of pancreatic cancer.Methods: A multi-institutional contouring dummy-run of two different cases of pancreatic cancer treated by postoperative and preoperative radiotherapy (RT) was performed. Clinical history, diagnostics, and planning CT imaging were available on AIRO website (http://www.radioterapiaitalia.it). Participants were requested to delineate CTVs according to their skills and knowledge. Aiming to quantify interobserver variability of CTVs delineations, the total volume, craniocaudal, laterolateral, and anteroposterior diameters were calculated. Descriptive statistic was calculated. The 95% Confidence Interval (95% CI) for coefficient of variation (CV) was estimated. The Dice Similarity Index (DSI) was used to evaluate the spatial overlap accuracy of the different CTVs compared with the CTVs of a national reference Centre considered as a benchmark. The mean DSI (mDSI) was calculated and reported.Results: A total of 18 radiation oncologists from different Institutes submitted the targets. Less variability was observed for the Elective CTV rather than the Boost CTV, in both cases. The estimated CV were 28.8% (95% CI: 21.2 - 45.0%) and 20.0% (95% CI: 14.9 - 30.6%) for the Elective CTV, in adjuvant (Case 1) and neoadjuvant (Case 2) case, respectively. The mDSI value was 0.68 for the Elective CTVs in both cases (range 0.19 - 0.79 in postoperative vs range 0.35 - 0.79 in preoperative case). The mDSI was increased to 0.71 (Case 1) and 0.72 (Case 2) if the observers with a worse agreement have been excluded. On the other hand, a CV of 42.4% (95% CI: 30.1 - 72.4%) and 63.8% (95% CI: 43.9 - 119.2%) with a mDSI value of 0.44 and 0.52, were calculated for the Boost CTV in Case 1 and Case 2, respectively.Conclusions: The CV and mDSI obtained values for Elective CTVs showed an acceptable agreement among participants either in postoperative as well in preoperative setting. Additional strategies to reduce the variability in Boost CTV delineation need to be found and promoted.

AB - Background: An observational multi-institutional study has been conducted aimed to evaluate the inter-observer variability in clinical target volume (CTV) delineation among different radiation oncologists in radiotherapy treatment of pancreatic cancer.Methods: A multi-institutional contouring dummy-run of two different cases of pancreatic cancer treated by postoperative and preoperative radiotherapy (RT) was performed. Clinical history, diagnostics, and planning CT imaging were available on AIRO website (http://www.radioterapiaitalia.it). Participants were requested to delineate CTVs according to their skills and knowledge. Aiming to quantify interobserver variability of CTVs delineations, the total volume, craniocaudal, laterolateral, and anteroposterior diameters were calculated. Descriptive statistic was calculated. The 95% Confidence Interval (95% CI) for coefficient of variation (CV) was estimated. The Dice Similarity Index (DSI) was used to evaluate the spatial overlap accuracy of the different CTVs compared with the CTVs of a national reference Centre considered as a benchmark. The mean DSI (mDSI) was calculated and reported.Results: A total of 18 radiation oncologists from different Institutes submitted the targets. Less variability was observed for the Elective CTV rather than the Boost CTV, in both cases. The estimated CV were 28.8% (95% CI: 21.2 - 45.0%) and 20.0% (95% CI: 14.9 - 30.6%) for the Elective CTV, in adjuvant (Case 1) and neoadjuvant (Case 2) case, respectively. The mDSI value was 0.68 for the Elective CTVs in both cases (range 0.19 - 0.79 in postoperative vs range 0.35 - 0.79 in preoperative case). The mDSI was increased to 0.71 (Case 1) and 0.72 (Case 2) if the observers with a worse agreement have been excluded. On the other hand, a CV of 42.4% (95% CI: 30.1 - 72.4%) and 63.8% (95% CI: 43.9 - 119.2%) with a mDSI value of 0.44 and 0.52, were calculated for the Boost CTV in Case 1 and Case 2, respectively.Conclusions: The CV and mDSI obtained values for Elective CTVs showed an acceptable agreement among participants either in postoperative as well in preoperative setting. Additional strategies to reduce the variability in Boost CTV delineation need to be found and promoted.

UR - http://www.scopus.com/inward/record.url?scp=84907045567&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84907045567&partnerID=8YFLogxK

U2 - 10.1186/1748-717X-9-198

DO - 10.1186/1748-717X-9-198

M3 - Article

C2 - 25199768

AN - SCOPUS:84907045567

VL - 9

JO - Radiation Oncology

JF - Radiation Oncology

SN - 1748-717X

IS - 1

M1 - 198

ER -