Performance of commercially available deformable image registration platforms for contour propagation using patient-based computational phantoms: A multi-institutional study

Gianfranco Loi, Marco Fusella, Eleonora Lanzi, Elisabetta Cagni, Cristina Garibaldi, Giuseppina Iacoviello, Francesco Lucio, Enrico Menghi, Roberto Miceli, Lucia C Orlandini, Antonella Roggio, Federica Rosica, Michele Stasi, Lidia Strigari, Silvia Strolin, Christian Fiandra

Research output: Contribution to journalArticle

Abstract

PURPOSE: To investigate the performance of various algorithms for deformable image registration (DIR) to propagate regions of interest (ROIs) using multiple commercial platforms.

METHODS AND MATERIALS: Thirteen institutions participated in the study with six commercial platforms: RayStation (RaySearch Laboratories, Stockholm, Sweden), MIM (Cleveland, OH, USA), VelocityAI and Smart Adapt (Varian Medical Systems, Palo Alto, CA, USA), Mirada XD (Mirada Medical Ltd, Oxford, UK), and ABAS (Elekta AB, Stockholm, Sweden). The DIR algorithms were tested on synthetic images generated with the ImSimQA package (Oncology Systems Limited, Shrewsbury, UK) by applying two specific Deformation Vector Fields (DVF) to real patient data-sets. Head-and-neck (HN), thorax, and pelvis sites were included. The accuracy of the algorithms was assessed by comparing the DIR-mapped ROIs from each center with those of reference, using the Dice Similarity Coefficient (DSC) and Mean Distance to Conformity (MDC) metrics. Statistical inference on validation results was carried out in order to identify the prognostic factors of DIR performances.

RESULTS: DVF intensity, anatomic site and participating center were significant prognostic factors of DIR performances. Sub-voxel accuracy was obtained in the HN by all algorithms. Large errors, with MDC ranging up to 6 mm, were observed in low-contrast regions that underwent significant deformation, such as in the pelvis, or large DVF with strong contrast, such as the clinical tumor volume (CTV) in the lung. Under these conditions, the hybrid DIR algorithms performed significantly better than the free-form intensity based algorithms and resulted robust against intercenter variability.

CONCLUSIONS: The performances of the systems proved to be site specific, depending on the DVF type and the platforms and the procedures used at the various centers. The pelvis was the most challenging site for most of the algorithms, which failed to achieve sub-voxel accuracy. Improved reproducibility was observed among the centers using the same hybrid registration algorithm.

Original languageEnglish
Pages (from-to)748-757
Number of pages10
JournalMedical Physics
Volume45
Issue number2
DOIs
Publication statusPublished - Feb 2018

Fingerprint

Pelvis
Sweden
Neck
Head
Tumor Burden
Thorax
Lung

Keywords

  • Journal Article

Cite this

Performance of commercially available deformable image registration platforms for contour propagation using patient-based computational phantoms : A multi-institutional study. / Loi, Gianfranco; Fusella, Marco; Lanzi, Eleonora; Cagni, Elisabetta; Garibaldi, Cristina; Iacoviello, Giuseppina; Lucio, Francesco; Menghi, Enrico; Miceli, Roberto; Orlandini, Lucia C; Roggio, Antonella; Rosica, Federica; Stasi, Michele; Strigari, Lidia; Strolin, Silvia; Fiandra, Christian.

In: Medical Physics, Vol. 45, No. 2, 02.2018, p. 748-757.

Research output: Contribution to journalArticle

Loi, Gianfranco ; Fusella, Marco ; Lanzi, Eleonora ; Cagni, Elisabetta ; Garibaldi, Cristina ; Iacoviello, Giuseppina ; Lucio, Francesco ; Menghi, Enrico ; Miceli, Roberto ; Orlandini, Lucia C ; Roggio, Antonella ; Rosica, Federica ; Stasi, Michele ; Strigari, Lidia ; Strolin, Silvia ; Fiandra, Christian. / Performance of commercially available deformable image registration platforms for contour propagation using patient-based computational phantoms : A multi-institutional study. In: Medical Physics. 2018 ; Vol. 45, No. 2. pp. 748-757.
@article{95ea6c6fbae642c582b3ad37bd3ce8eb,
title = "Performance of commercially available deformable image registration platforms for contour propagation using patient-based computational phantoms: A multi-institutional study",
abstract = "PURPOSE: To investigate the performance of various algorithms for deformable image registration (DIR) to propagate regions of interest (ROIs) using multiple commercial platforms.METHODS AND MATERIALS: Thirteen institutions participated in the study with six commercial platforms: RayStation (RaySearch Laboratories, Stockholm, Sweden), MIM (Cleveland, OH, USA), VelocityAI and Smart Adapt (Varian Medical Systems, Palo Alto, CA, USA), Mirada XD (Mirada Medical Ltd, Oxford, UK), and ABAS (Elekta AB, Stockholm, Sweden). The DIR algorithms were tested on synthetic images generated with the ImSimQA package (Oncology Systems Limited, Shrewsbury, UK) by applying two specific Deformation Vector Fields (DVF) to real patient data-sets. Head-and-neck (HN), thorax, and pelvis sites were included. The accuracy of the algorithms was assessed by comparing the DIR-mapped ROIs from each center with those of reference, using the Dice Similarity Coefficient (DSC) and Mean Distance to Conformity (MDC) metrics. Statistical inference on validation results was carried out in order to identify the prognostic factors of DIR performances.RESULTS: DVF intensity, anatomic site and participating center were significant prognostic factors of DIR performances. Sub-voxel accuracy was obtained in the HN by all algorithms. Large errors, with MDC ranging up to 6 mm, were observed in low-contrast regions that underwent significant deformation, such as in the pelvis, or large DVF with strong contrast, such as the clinical tumor volume (CTV) in the lung. Under these conditions, the hybrid DIR algorithms performed significantly better than the free-form intensity based algorithms and resulted robust against intercenter variability.CONCLUSIONS: The performances of the systems proved to be site specific, depending on the DVF type and the platforms and the procedures used at the various centers. The pelvis was the most challenging site for most of the algorithms, which failed to achieve sub-voxel accuracy. Improved reproducibility was observed among the centers using the same hybrid registration algorithm.",
keywords = "Journal Article",
author = "Gianfranco Loi and Marco Fusella and Eleonora Lanzi and Elisabetta Cagni and Cristina Garibaldi and Giuseppina Iacoviello and Francesco Lucio and Enrico Menghi and Roberto Miceli and Orlandini, {Lucia C} and Antonella Roggio and Federica Rosica and Michele Stasi and Lidia Strigari and Silvia Strolin and Christian Fiandra",
note = "{\circledC} 2017 American Association of Physicists in Medicine.",
year = "2018",
month = "2",
doi = "10.1002/mp.12737",
language = "English",
volume = "45",
pages = "748--757",
journal = "Medical Physics",
issn = "0094-2405",
publisher = "John Wiley and Sons Ltd",
number = "2",

}

TY - JOUR

T1 - Performance of commercially available deformable image registration platforms for contour propagation using patient-based computational phantoms

T2 - A multi-institutional study

AU - Loi, Gianfranco

AU - Fusella, Marco

AU - Lanzi, Eleonora

AU - Cagni, Elisabetta

AU - Garibaldi, Cristina

AU - Iacoviello, Giuseppina

AU - Lucio, Francesco

AU - Menghi, Enrico

AU - Miceli, Roberto

AU - Orlandini, Lucia C

AU - Roggio, Antonella

AU - Rosica, Federica

AU - Stasi, Michele

AU - Strigari, Lidia

AU - Strolin, Silvia

AU - Fiandra, Christian

N1 - © 2017 American Association of Physicists in Medicine.

PY - 2018/2

Y1 - 2018/2

N2 - PURPOSE: To investigate the performance of various algorithms for deformable image registration (DIR) to propagate regions of interest (ROIs) using multiple commercial platforms.METHODS AND MATERIALS: Thirteen institutions participated in the study with six commercial platforms: RayStation (RaySearch Laboratories, Stockholm, Sweden), MIM (Cleveland, OH, USA), VelocityAI and Smart Adapt (Varian Medical Systems, Palo Alto, CA, USA), Mirada XD (Mirada Medical Ltd, Oxford, UK), and ABAS (Elekta AB, Stockholm, Sweden). The DIR algorithms were tested on synthetic images generated with the ImSimQA package (Oncology Systems Limited, Shrewsbury, UK) by applying two specific Deformation Vector Fields (DVF) to real patient data-sets. Head-and-neck (HN), thorax, and pelvis sites were included. The accuracy of the algorithms was assessed by comparing the DIR-mapped ROIs from each center with those of reference, using the Dice Similarity Coefficient (DSC) and Mean Distance to Conformity (MDC) metrics. Statistical inference on validation results was carried out in order to identify the prognostic factors of DIR performances.RESULTS: DVF intensity, anatomic site and participating center were significant prognostic factors of DIR performances. Sub-voxel accuracy was obtained in the HN by all algorithms. Large errors, with MDC ranging up to 6 mm, were observed in low-contrast regions that underwent significant deformation, such as in the pelvis, or large DVF with strong contrast, such as the clinical tumor volume (CTV) in the lung. Under these conditions, the hybrid DIR algorithms performed significantly better than the free-form intensity based algorithms and resulted robust against intercenter variability.CONCLUSIONS: The performances of the systems proved to be site specific, depending on the DVF type and the platforms and the procedures used at the various centers. The pelvis was the most challenging site for most of the algorithms, which failed to achieve sub-voxel accuracy. Improved reproducibility was observed among the centers using the same hybrid registration algorithm.

AB - PURPOSE: To investigate the performance of various algorithms for deformable image registration (DIR) to propagate regions of interest (ROIs) using multiple commercial platforms.METHODS AND MATERIALS: Thirteen institutions participated in the study with six commercial platforms: RayStation (RaySearch Laboratories, Stockholm, Sweden), MIM (Cleveland, OH, USA), VelocityAI and Smart Adapt (Varian Medical Systems, Palo Alto, CA, USA), Mirada XD (Mirada Medical Ltd, Oxford, UK), and ABAS (Elekta AB, Stockholm, Sweden). The DIR algorithms were tested on synthetic images generated with the ImSimQA package (Oncology Systems Limited, Shrewsbury, UK) by applying two specific Deformation Vector Fields (DVF) to real patient data-sets. Head-and-neck (HN), thorax, and pelvis sites were included. The accuracy of the algorithms was assessed by comparing the DIR-mapped ROIs from each center with those of reference, using the Dice Similarity Coefficient (DSC) and Mean Distance to Conformity (MDC) metrics. Statistical inference on validation results was carried out in order to identify the prognostic factors of DIR performances.RESULTS: DVF intensity, anatomic site and participating center were significant prognostic factors of DIR performances. Sub-voxel accuracy was obtained in the HN by all algorithms. Large errors, with MDC ranging up to 6 mm, were observed in low-contrast regions that underwent significant deformation, such as in the pelvis, or large DVF with strong contrast, such as the clinical tumor volume (CTV) in the lung. Under these conditions, the hybrid DIR algorithms performed significantly better than the free-form intensity based algorithms and resulted robust against intercenter variability.CONCLUSIONS: The performances of the systems proved to be site specific, depending on the DVF type and the platforms and the procedures used at the various centers. The pelvis was the most challenging site for most of the algorithms, which failed to achieve sub-voxel accuracy. Improved reproducibility was observed among the centers using the same hybrid registration algorithm.

KW - Journal Article

U2 - 10.1002/mp.12737

DO - 10.1002/mp.12737

M3 - Article

C2 - 29266262

VL - 45

SP - 748

EP - 757

JO - Medical Physics

JF - Medical Physics

SN - 0094-2405

IS - 2

ER -