Deep action learning enables robust 3D segmentation of body organs in various CT and MRI images

Abstract In this study, we propose a novel point cloud based 3D registration and segmentation framework using reinforcement learning. An artificial agent, implemented as a distinct actor based on value networks, is trained to predict the optimal piece-wise linear transformation of a point cloud for...

Descripción completa

Guardado en:

Detalles Bibliográficos
Autores principales:	Xia Zhong, Mario Amrehn, Nishant Ravikumar, Shuqing Chen, Norbert Strobel, Annette Birkhold, Markus Kowarschik, Rebecca Fahrig, Andreas Maier
Formato:	article
Lenguaje:	EN
Publicado:	Nature Portfolio 2021
Materias:	Medicine R Science Q
Acceso en línea:	https://doaj.org/article/7b70a3b7650e4ff28f63408369c0ddec
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

id	oai:doaj.org-article:7b70a3b7650e4ff28f63408369c0ddec
record_format	dspace
spelling	oai:doaj.org-article:7b70a3b7650e4ff28f63408369c0ddec2021-12-02T14:26:55ZDeep action learning enables robust 3D segmentation of body organs in various CT and MRI images10.1038/s41598-021-82370-62045-2322https://doaj.org/article/7b70a3b7650e4ff28f63408369c0ddec2021-02-01T00:00:00Zhttps://doi.org/10.1038/s41598-021-82370-6https://doaj.org/toc/2045-2322Abstract In this study, we propose a novel point cloud based 3D registration and segmentation framework using reinforcement learning. An artificial agent, implemented as a distinct actor based on value networks, is trained to predict the optimal piece-wise linear transformation of a point cloud for the joint tasks of registration and segmentation. The actor network estimates a set of plausible actions and the value network aims to select the optimal action for the current observation. Point-wise features that comprise spatial positions (and surface normal vectors in the case of structured meshes), and their corresponding image features, are used to encode the observation and represent the underlying 3D volume. The actor and value networks are applied iteratively to estimate a sequence of transformations that enable accurate delineation of object boundaries. The proposed approach was extensively evaluated in both segmentation and registration tasks using a variety of challenging clinical datasets. Our method has fewer trainable parameters and lower computational complexity compared to the 3D U-Net, and it is independent of the volume resolution. We show that the proposed method is applicable to mono- and multi-modal segmentation tasks, achieving significant improvements over the state-of-the-art for the latter. The flexibility of the proposed framework is further demonstrated for a multi-modal registration application. As we learn to predict actions rather than a target, the proposed method is more robust compared to the 3D U-Net when dealing with previously unseen datasets, acquired using different protocols or modalities. As a result, the proposed method provides a promising multi-purpose segmentation and registration framework, particular in the context of image-guided interventions.Xia ZhongMario AmrehnNishant RavikumarShuqing ChenNorbert StrobelAnnette BirkholdMarkus KowarschikRebecca FahrigAndreas MaierNature PortfolioarticleMedicineRScienceQENScientific Reports, Vol 11, Iss 1, Pp 1-14 (2021)
institution	DOAJ
collection	DOAJ
language	EN
topic	Medicine R Science Q
spellingShingle	Medicine R Science Q Xia Zhong Mario Amrehn Nishant Ravikumar Shuqing Chen Norbert Strobel Annette Birkhold Markus Kowarschik Rebecca Fahrig Andreas Maier Deep action learning enables robust 3D segmentation of body organs in various CT and MRI images
description	Abstract In this study, we propose a novel point cloud based 3D registration and segmentation framework using reinforcement learning. An artificial agent, implemented as a distinct actor based on value networks, is trained to predict the optimal piece-wise linear transformation of a point cloud for the joint tasks of registration and segmentation. The actor network estimates a set of plausible actions and the value network aims to select the optimal action for the current observation. Point-wise features that comprise spatial positions (and surface normal vectors in the case of structured meshes), and their corresponding image features, are used to encode the observation and represent the underlying 3D volume. The actor and value networks are applied iteratively to estimate a sequence of transformations that enable accurate delineation of object boundaries. The proposed approach was extensively evaluated in both segmentation and registration tasks using a variety of challenging clinical datasets. Our method has fewer trainable parameters and lower computational complexity compared to the 3D U-Net, and it is independent of the volume resolution. We show that the proposed method is applicable to mono- and multi-modal segmentation tasks, achieving significant improvements over the state-of-the-art for the latter. The flexibility of the proposed framework is further demonstrated for a multi-modal registration application. As we learn to predict actions rather than a target, the proposed method is more robust compared to the 3D U-Net when dealing with previously unseen datasets, acquired using different protocols or modalities. As a result, the proposed method provides a promising multi-purpose segmentation and registration framework, particular in the context of image-guided interventions.
format	article
author	Xia Zhong Mario Amrehn Nishant Ravikumar Shuqing Chen Norbert Strobel Annette Birkhold Markus Kowarschik Rebecca Fahrig Andreas Maier
author_facet	Xia Zhong Mario Amrehn Nishant Ravikumar Shuqing Chen Norbert Strobel Annette Birkhold Markus Kowarschik Rebecca Fahrig Andreas Maier
author_sort	Xia Zhong
title	Deep action learning enables robust 3D segmentation of body organs in various CT and MRI images
title_short	Deep action learning enables robust 3D segmentation of body organs in various CT and MRI images
title_full	Deep action learning enables robust 3D segmentation of body organs in various CT and MRI images
title_fullStr	Deep action learning enables robust 3D segmentation of body organs in various CT and MRI images
title_full_unstemmed	Deep action learning enables robust 3D segmentation of body organs in various CT and MRI images
title_sort	deep action learning enables robust 3d segmentation of body organs in various ct and mri images
publisher	Nature Portfolio
publishDate	2021
url	https://doaj.org/article/7b70a3b7650e4ff28f63408369c0ddec
work_keys_str_mv	AT xiazhong deepactionlearningenablesrobust3dsegmentationofbodyorgansinvariousctandmriimages AT marioamrehn deepactionlearningenablesrobust3dsegmentationofbodyorgansinvariousctandmriimages AT nishantravikumar deepactionlearningenablesrobust3dsegmentationofbodyorgansinvariousctandmriimages AT shuqingchen deepactionlearningenablesrobust3dsegmentationofbodyorgansinvariousctandmriimages AT norbertstrobel deepactionlearningenablesrobust3dsegmentationofbodyorgansinvariousctandmriimages AT annettebirkhold deepactionlearningenablesrobust3dsegmentationofbodyorgansinvariousctandmriimages AT markuskowarschik deepactionlearningenablesrobust3dsegmentationofbodyorgansinvariousctandmriimages AT rebeccafahrig deepactionlearningenablesrobust3dsegmentationofbodyorgansinvariousctandmriimages AT andreasmaier deepactionlearningenablesrobust3dsegmentationofbodyorgansinvariousctandmriimages
_version_	1718391319591649280

Deep action learning enables robust 3D segmentation of body organs in various CT and MRI images

Ejemplares similares