TY - JOUR
T1 - Towards realistic laparoscopic image generation using image-domain translation
AU - Marzullo, Aldo
AU - Moccia, Sara
AU - Catellani, Michele
AU - Calimeri, Francesco
AU - Momi, Elena De
N1 - Funding Information:
This study did not need any ethical approval. The authors declare no competing interests. The study was partially supported by the European Commission and the Regione Calabria within the program POR Calabria FESR-FSE 2014/2020 , topic “ICT e Terziario Innovativo”. We gratefully acknowledge the support of NVIDIA Corporation with the donation of the Quadro P6000 GPU used for this research.
Publisher Copyright:
© 2020
PY - 2021/3
Y1 - 2021/3
N2 - Background and ObjectivesOver the last decade, Deep Learning (DL) has revolutionized data analysis in many areas, including medical imaging. However, there is a bottleneck in the advancement of DL in the surgery field, which can be seen in a shortage of large-scale data, which in turn may be attributed to the lack of a structured and standardized methodology for storing and analyzing surgical images in clinical centres. Furthermore, accurate annotations manually added are expensive and time consuming. A great help can come from the synthesis of artificial images; in this context, in the latest years, the use of Generative Adversarial Neural Networks (GANs) achieved promising results in obtaining photo-realistic images. MethodsIn this study, a method for Minimally Invasive Surgery (MIS) image synthesis is proposed. To this aim, the generative adversarial network pix2pix is trained to generate paired annotated MIS images by transforming rough segmentation of surgical instruments and tissues into realistic images. An additional regularization term was added to the original optimization problem, in order to enhance realism of surgical tools with respect to the background. Results Quantitative and qualitative (i.e., human-based) evaluations of generated images have been carried out in order to assess the effectiveness of the method. ConclusionsExperimental results show that the proposed method is actually able to translate MIS segmentations to realistic MIS images, which can in turn be used to augment existing data sets and help at overcoming the lack of useful images; this allows physicians and algorithms to take advantage from new annotated instances for their training.
AB - Background and ObjectivesOver the last decade, Deep Learning (DL) has revolutionized data analysis in many areas, including medical imaging. However, there is a bottleneck in the advancement of DL in the surgery field, which can be seen in a shortage of large-scale data, which in turn may be attributed to the lack of a structured and standardized methodology for storing and analyzing surgical images in clinical centres. Furthermore, accurate annotations manually added are expensive and time consuming. A great help can come from the synthesis of artificial images; in this context, in the latest years, the use of Generative Adversarial Neural Networks (GANs) achieved promising results in obtaining photo-realistic images. MethodsIn this study, a method for Minimally Invasive Surgery (MIS) image synthesis is proposed. To this aim, the generative adversarial network pix2pix is trained to generate paired annotated MIS images by transforming rough segmentation of surgical instruments and tissues into realistic images. An additional regularization term was added to the original optimization problem, in order to enhance realism of surgical tools with respect to the background. Results Quantitative and qualitative (i.e., human-based) evaluations of generated images have been carried out in order to assess the effectiveness of the method. ConclusionsExperimental results show that the proposed method is actually able to translate MIS segmentations to realistic MIS images, which can in turn be used to augment existing data sets and help at overcoming the lack of useful images; this allows physicians and algorithms to take advantage from new annotated instances for their training.
KW - Data Augmentation
KW - Generative Adversarial Networks
KW - Image translation
KW - Minimally Invasive Surgery
UR - http://www.scopus.com/inward/record.url?scp=85096562492&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85096562492&partnerID=8YFLogxK
U2 - 10.1016/j.cmpb.2020.105834
DO - 10.1016/j.cmpb.2020.105834
M3 - Article
C2 - 33229016
AN - SCOPUS:85096562492
VL - 200
JO - Computer Methods and Programs in Biomedicine
JF - Computer Methods and Programs in Biomedicine
SN - 0169-2607
M1 - 105834
ER -