Pareto-optimal plans as ground truth for validation of a commercial system for knowledge-based DVH-prediction

Elisabetta Cagni, Andrea Botti, Yibing Wang, Mauro Iori, Steven F Petit, Ben J M Heijmen

Research output: Contribution to journalArticle

Abstract

PURPOSE: Treatment plans manually generated in clinical routine may suffer from variations and inconsistencies in quality. Using such plans for validating a DVH prediction algorithm might obscure its intrinsic prediction accuracy. In this study we used a recently published large database of Pareto-optimal prostate cancer plans to assess the prediction accuracy of a commercial knowledge-based DVH prediction algorithm, RapidPlan. The database plans were consistently generated with automated planning using an independent optimizer, and can be considered as aground truth of plan quality.

METHODS: Prediction models were generated using training sets with 20, 30, 45, 55 and 114 Pareto-optimal plans. Model-20 and Model-30 were built using 5 groups of randomly selected training patients. For 60 independent Pareto-optimal validation plans, predicted and database DVHs were compared.

RESULTS: For model-114, differences between predicted and database mean doses of more than ± 10% in rectum, anus and bladder, occurred for 23.3%, 55.0%, and 6.7% of the validation plans, respectively. For rectum V65Gy and V75Gy, differences outside the ±10% range were observed in 21.7% and 70.0% of validation plans, respectively. For 61.7% of validation plans, inaccuracies in predicted rectum DVHs resulted in a deviation in predicted NTCP for rectal bleeding outside ±10%. With smaller training sets the DVH prediction performance deteriorated, showing dependence on the selected training patients.

CONCLUSION: Even when analysed with Pareto-optimal plans with highly consistent quality, clinically relevant deviations in DVH predictions were observed. Such deviations could potentially result in suboptimal plans for new patients. Further research on DVH prediction models is warranted.

Original languageEnglish
Pages (from-to)98-106
Number of pages9
JournalPhysica Medica
Volume55
DOIs
Publication statusPublished - Nov 2018

Fingerprint

ground truth
Rectum
Databases
predictions
rectum
Anal Canal
education
Prostatic Neoplasms
Urinary Bladder
Hemorrhage
deviation
Research
bleeding
performance prediction
bladder
planning
Therapeutics
cancer

Keywords

  • Humans
  • Male
  • Prostatic Neoplasms/radiotherapy
  • Radiation Dosage
  • Radiotherapy Dosage
  • Radiotherapy Planning, Computer-Assisted/methods
  • Radiotherapy, Intensity-Modulated

Cite this

Pareto-optimal plans as ground truth for validation of a commercial system for knowledge-based DVH-prediction. / Cagni, Elisabetta; Botti, Andrea; Wang, Yibing; Iori, Mauro; Petit, Steven F; Heijmen, Ben J M.

In: Physica Medica, Vol. 55, 11.2018, p. 98-106.

Research output: Contribution to journalArticle

Cagni, Elisabetta ; Botti, Andrea ; Wang, Yibing ; Iori, Mauro ; Petit, Steven F ; Heijmen, Ben J M. / Pareto-optimal plans as ground truth for validation of a commercial system for knowledge-based DVH-prediction. In: Physica Medica. 2018 ; Vol. 55. pp. 98-106.
@article{e4f7c65cd1a84536bbfad5a98c3da360,
title = "Pareto-optimal plans as ground truth for validation of a commercial system for knowledge-based DVH-prediction",
abstract = "PURPOSE: Treatment plans manually generated in clinical routine may suffer from variations and inconsistencies in quality. Using such plans for validating a DVH prediction algorithm might obscure its intrinsic prediction accuracy. In this study we used a recently published large database of Pareto-optimal prostate cancer plans to assess the prediction accuracy of a commercial knowledge-based DVH prediction algorithm, RapidPlan. The database plans were consistently generated with automated planning using an independent optimizer, and can be considered as aground truth of plan quality.METHODS: Prediction models were generated using training sets with 20, 30, 45, 55 and 114 Pareto-optimal plans. Model-20 and Model-30 were built using 5 groups of randomly selected training patients. For 60 independent Pareto-optimal validation plans, predicted and database DVHs were compared.RESULTS: For model-114, differences between predicted and database mean doses of more than ± 10{\%} in rectum, anus and bladder, occurred for 23.3{\%}, 55.0{\%}, and 6.7{\%} of the validation plans, respectively. For rectum V65Gy and V75Gy, differences outside the ±10{\%} range were observed in 21.7{\%} and 70.0{\%} of validation plans, respectively. For 61.7{\%} of validation plans, inaccuracies in predicted rectum DVHs resulted in a deviation in predicted NTCP for rectal bleeding outside ±10{\%}. With smaller training sets the DVH prediction performance deteriorated, showing dependence on the selected training patients.CONCLUSION: Even when analysed with Pareto-optimal plans with highly consistent quality, clinically relevant deviations in DVH predictions were observed. Such deviations could potentially result in suboptimal plans for new patients. Further research on DVH prediction models is warranted.",
keywords = "Humans, Male, Prostatic Neoplasms/radiotherapy, Radiation Dosage, Radiotherapy Dosage, Radiotherapy Planning, Computer-Assisted/methods, Radiotherapy, Intensity-Modulated",
author = "Elisabetta Cagni and Andrea Botti and Yibing Wang and Mauro Iori and Petit, {Steven F} and Heijmen, {Ben J M}",
note = "Copyright {\circledC} 2018. Published by Elsevier Ltd.",
year = "2018",
month = "11",
doi = "10.1016/j.ejmp.2018.11.002",
language = "English",
volume = "55",
pages = "98--106",
journal = "Physica Medica",
issn = "1120-1797",
publisher = "Associazione Italiana di Fisica Medica",

}

TY - JOUR

T1 - Pareto-optimal plans as ground truth for validation of a commercial system for knowledge-based DVH-prediction

AU - Cagni, Elisabetta

AU - Botti, Andrea

AU - Wang, Yibing

AU - Iori, Mauro

AU - Petit, Steven F

AU - Heijmen, Ben J M

N1 - Copyright © 2018. Published by Elsevier Ltd.

PY - 2018/11

Y1 - 2018/11

N2 - PURPOSE: Treatment plans manually generated in clinical routine may suffer from variations and inconsistencies in quality. Using such plans for validating a DVH prediction algorithm might obscure its intrinsic prediction accuracy. In this study we used a recently published large database of Pareto-optimal prostate cancer plans to assess the prediction accuracy of a commercial knowledge-based DVH prediction algorithm, RapidPlan. The database plans were consistently generated with automated planning using an independent optimizer, and can be considered as aground truth of plan quality.METHODS: Prediction models were generated using training sets with 20, 30, 45, 55 and 114 Pareto-optimal plans. Model-20 and Model-30 were built using 5 groups of randomly selected training patients. For 60 independent Pareto-optimal validation plans, predicted and database DVHs were compared.RESULTS: For model-114, differences between predicted and database mean doses of more than ± 10% in rectum, anus and bladder, occurred for 23.3%, 55.0%, and 6.7% of the validation plans, respectively. For rectum V65Gy and V75Gy, differences outside the ±10% range were observed in 21.7% and 70.0% of validation plans, respectively. For 61.7% of validation plans, inaccuracies in predicted rectum DVHs resulted in a deviation in predicted NTCP for rectal bleeding outside ±10%. With smaller training sets the DVH prediction performance deteriorated, showing dependence on the selected training patients.CONCLUSION: Even when analysed with Pareto-optimal plans with highly consistent quality, clinically relevant deviations in DVH predictions were observed. Such deviations could potentially result in suboptimal plans for new patients. Further research on DVH prediction models is warranted.

AB - PURPOSE: Treatment plans manually generated in clinical routine may suffer from variations and inconsistencies in quality. Using such plans for validating a DVH prediction algorithm might obscure its intrinsic prediction accuracy. In this study we used a recently published large database of Pareto-optimal prostate cancer plans to assess the prediction accuracy of a commercial knowledge-based DVH prediction algorithm, RapidPlan. The database plans were consistently generated with automated planning using an independent optimizer, and can be considered as aground truth of plan quality.METHODS: Prediction models were generated using training sets with 20, 30, 45, 55 and 114 Pareto-optimal plans. Model-20 and Model-30 were built using 5 groups of randomly selected training patients. For 60 independent Pareto-optimal validation plans, predicted and database DVHs were compared.RESULTS: For model-114, differences between predicted and database mean doses of more than ± 10% in rectum, anus and bladder, occurred for 23.3%, 55.0%, and 6.7% of the validation plans, respectively. For rectum V65Gy and V75Gy, differences outside the ±10% range were observed in 21.7% and 70.0% of validation plans, respectively. For 61.7% of validation plans, inaccuracies in predicted rectum DVHs resulted in a deviation in predicted NTCP for rectal bleeding outside ±10%. With smaller training sets the DVH prediction performance deteriorated, showing dependence on the selected training patients.CONCLUSION: Even when analysed with Pareto-optimal plans with highly consistent quality, clinically relevant deviations in DVH predictions were observed. Such deviations could potentially result in suboptimal plans for new patients. Further research on DVH prediction models is warranted.

KW - Humans

KW - Male

KW - Prostatic Neoplasms/radiotherapy

KW - Radiation Dosage

KW - Radiotherapy Dosage

KW - Radiotherapy Planning, Computer-Assisted/methods

KW - Radiotherapy, Intensity-Modulated

U2 - 10.1016/j.ejmp.2018.11.002

DO - 10.1016/j.ejmp.2018.11.002

M3 - Article

C2 - 30471826

VL - 55

SP - 98

EP - 106

JO - Physica Medica

JF - Physica Medica

SN - 1120-1797

ER -