Temporal refinement of 3D CNN semantic segmentations on 4D time-series of undersampled tomograms using hidden Markov models

Abstract Recently, several convolutional neural networks have been proposed not only for 2D images, but also for 3D and 4D volume segmentation. Nevertheless, due to the large data size of the latter, acquiring a sufficient amount of training annotations is much more strenuous than in 2D images. For...

Descripción completa

Guardado en:

Detalles Bibliográficos
Autores principales:	Dimitrios Bellos, Mark Basham, Tony Pridmore, Andrew P. French
Formato:	article
Lenguaje:	EN
Publicado:	Nature Portfolio 2021
Materias:	Medicine R Science Q
Acceso en línea:	https://doaj.org/article/a20a76b6c52f42b2aa60d53e45e0824b
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

id	oai:doaj.org-article:a20a76b6c52f42b2aa60d53e45e0824b
record_format	dspace
spelling	oai:doaj.org-article:a20a76b6c52f42b2aa60d53e45e0824b2021-12-05T12:15:54ZTemporal refinement of 3D CNN semantic segmentations on 4D time-series of undersampled tomograms using hidden Markov models10.1038/s41598-021-02466-x2045-2322https://doaj.org/article/a20a76b6c52f42b2aa60d53e45e0824b2021-12-01T00:00:00Zhttps://doi.org/10.1038/s41598-021-02466-xhttps://doaj.org/toc/2045-2322Abstract Recently, several convolutional neural networks have been proposed not only for 2D images, but also for 3D and 4D volume segmentation. Nevertheless, due to the large data size of the latter, acquiring a sufficient amount of training annotations is much more strenuous than in 2D images. For 4D time-series tomograms, this is usually handled by segmenting the constituent tomograms independently through time with 3D convolutional neural networks. Inter-volume information is therefore not utilized, potentially leading to temporal incoherence. In this paper, we attempt to resolve this by proposing two hidden Markov model variants that refine 4D segmentation labels made by 3D convolutional neural networks working on each time point. Our models utilize not only inter-volume information, but also the prediction confidence generated by the 3D segmentation convolutional neural networks themselves. To the best of our knowledge, this is the first attempt to refine 4D segmentations made by 3D convolutional neural networks using hidden Markov models. During experiments we test our models, qualitatively, quantitatively and behaviourally, using prespecified segmentations. We demonstrate in the domain of time series tomograms which are typically undersampled to allow more frequent capture; a particularly challenging problem. Finally, our dataset and code is publicly available.Dimitrios BellosMark BashamTony PridmoreAndrew P. FrenchNature PortfolioarticleMedicineRScienceQENScientific Reports, Vol 11, Iss 1, Pp 1-14 (2021)
institution	DOAJ
collection	DOAJ
language	EN
topic	Medicine R Science Q
spellingShingle	Medicine R Science Q Dimitrios Bellos Mark Basham Tony Pridmore Andrew P. French Temporal refinement of 3D CNN semantic segmentations on 4D time-series of undersampled tomograms using hidden Markov models
description	Abstract Recently, several convolutional neural networks have been proposed not only for 2D images, but also for 3D and 4D volume segmentation. Nevertheless, due to the large data size of the latter, acquiring a sufficient amount of training annotations is much more strenuous than in 2D images. For 4D time-series tomograms, this is usually handled by segmenting the constituent tomograms independently through time with 3D convolutional neural networks. Inter-volume information is therefore not utilized, potentially leading to temporal incoherence. In this paper, we attempt to resolve this by proposing two hidden Markov model variants that refine 4D segmentation labels made by 3D convolutional neural networks working on each time point. Our models utilize not only inter-volume information, but also the prediction confidence generated by the 3D segmentation convolutional neural networks themselves. To the best of our knowledge, this is the first attempt to refine 4D segmentations made by 3D convolutional neural networks using hidden Markov models. During experiments we test our models, qualitatively, quantitatively and behaviourally, using prespecified segmentations. We demonstrate in the domain of time series tomograms which are typically undersampled to allow more frequent capture; a particularly challenging problem. Finally, our dataset and code is publicly available.
format	article
author	Dimitrios Bellos Mark Basham Tony Pridmore Andrew P. French
author_facet	Dimitrios Bellos Mark Basham Tony Pridmore Andrew P. French
author_sort	Dimitrios Bellos
title	Temporal refinement of 3D CNN semantic segmentations on 4D time-series of undersampled tomograms using hidden Markov models
title_short	Temporal refinement of 3D CNN semantic segmentations on 4D time-series of undersampled tomograms using hidden Markov models
title_full	Temporal refinement of 3D CNN semantic segmentations on 4D time-series of undersampled tomograms using hidden Markov models
title_fullStr	Temporal refinement of 3D CNN semantic segmentations on 4D time-series of undersampled tomograms using hidden Markov models
title_full_unstemmed	Temporal refinement of 3D CNN semantic segmentations on 4D time-series of undersampled tomograms using hidden Markov models
title_sort	temporal refinement of 3d cnn semantic segmentations on 4d time-series of undersampled tomograms using hidden markov models
publisher	Nature Portfolio
publishDate	2021
url	https://doaj.org/article/a20a76b6c52f42b2aa60d53e45e0824b
work_keys_str_mv	AT dimitriosbellos temporalrefinementof3dcnnsemanticsegmentationson4dtimeseriesofundersampledtomogramsusinghiddenmarkovmodels AT markbasham temporalrefinementof3dcnnsemanticsegmentationson4dtimeseriesofundersampledtomogramsusinghiddenmarkovmodels AT tonypridmore temporalrefinementof3dcnnsemanticsegmentationson4dtimeseriesofundersampledtomogramsusinghiddenmarkovmodels AT andrewpfrench temporalrefinementof3dcnnsemanticsegmentationson4dtimeseriesofundersampledtomogramsusinghiddenmarkovmodels
_version_	1718372091730853888

Temporal refinement of 3D CNN semantic segmentations on 4D time-series of undersampled tomograms using hidden Markov models

Ejemplares similares