Multimodal Medical Image Fusion Based on Gabor Representation Combination of Multi-CNN and Fuzzy Neural Network

Aiming at the current multimodal medical image fusion methods that cannot fully characterize the complex textures and edge information of the lesion in the fused image, a method based on Gabor representation of multi-CNN combination and fuzzy neural network is proposed. This method first filters the...

Descripción completa

Guardado en:

Detalles Bibliográficos
Autores principales:	Lifang Wang, Jin Zhang, Yang Liu, Jia Mi, Jiong Zhang
Formato:	article
Lenguaje:	EN
Publicado:	IEEE 2021
Materias:	Medical image fusion G-CNNs Gabor representation convolutional neural network fuzzy neural network Electrical engineering. Electronics. Nuclear engineering TK1-9971
Acceso en línea:	https://doaj.org/article/c56b7a66d70d45ce82060e474d890372
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

id	oai:doaj.org-article:c56b7a66d70d45ce82060e474d890372
record_format	dspace
spelling	oai:doaj.org-article:c56b7a66d70d45ce82060e474d8903722021-11-19T00:06:32ZMultimodal Medical Image Fusion Based on Gabor Representation Combination of Multi-CNN and Fuzzy Neural Network2169-353610.1109/ACCESS.2021.3075953https://doaj.org/article/c56b7a66d70d45ce82060e474d8903722021-01-01T00:00:00Zhttps://ieeexplore.ieee.org/document/9416689/https://doaj.org/toc/2169-3536Aiming at the current multimodal medical image fusion methods that cannot fully characterize the complex textures and edge information of the lesion in the fused image, a method based on Gabor representation of multi-CNN combination and fuzzy neural network is proposed. This method first filters the CT and MR image sets through a set of Gabor filter banks with different proportions and directions to obtain different Gabor representations pairs of CT and MR, each pair of different Gabor representations is used to train the corresponding CNN to generate a G- CNN and multiple G- CNN form a G- CNN group, namely G- CNNs; then when fusing CT and MR images, CT and MR are represented by Gabors to get Gabor representation pairs firstly, each Gabor representation pair is put into the corresponding trained G- CNN for preliminary fusion, then use the fuzzy neural network to fuse multiple outputs of the G- CNNs to obtain the final fused image. Compared with the nine recent state-of-the-art multimodal fusion methods, the average mutual information of the three groups of experiments has increased by 13%, 10.3%, and 10% respectively; the average spatial frequency has increased by 10.3%, 20%, and 10.7%; the average standard deviation has increased respectively 12.4%, 10.8%, 14.4%; the average edge retention information increased by 33.5%, 22%, and 43%. The experimental results show that the proposed fusion method is significantly better than the other comparative fusion methods in objective evaluation and visual quality. It has the best performance on the four indicators and can better integrate the rich texture features and the clear edge information of the source images into the final fused image, which improves the quality of multimodal medical image fusion, and effectively assists doctors in disease diagnosis.Lifang WangJin ZhangYang LiuJia MiJiong ZhangIEEEarticleMedical image fusionG-CNNsGabor representationconvolutional neural networkfuzzy neural networkElectrical engineering. Electronics. Nuclear engineeringTK1-9971ENIEEE Access, Vol 9, Pp 67634-67647 (2021)
institution	DOAJ
collection	DOAJ
language	EN
topic	Medical image fusion G-CNNs Gabor representation convolutional neural network fuzzy neural network Electrical engineering. Electronics. Nuclear engineering TK1-9971
spellingShingle	Medical image fusion G-CNNs Gabor representation convolutional neural network fuzzy neural network Electrical engineering. Electronics. Nuclear engineering TK1-9971 Lifang Wang Jin Zhang Yang Liu Jia Mi Jiong Zhang Multimodal Medical Image Fusion Based on Gabor Representation Combination of Multi-CNN and Fuzzy Neural Network
description	Aiming at the current multimodal medical image fusion methods that cannot fully characterize the complex textures and edge information of the lesion in the fused image, a method based on Gabor representation of multi-CNN combination and fuzzy neural network is proposed. This method first filters the CT and MR image sets through a set of Gabor filter banks with different proportions and directions to obtain different Gabor representations pairs of CT and MR, each pair of different Gabor representations is used to train the corresponding CNN to generate a G- CNN and multiple G- CNN form a G- CNN group, namely G- CNNs; then when fusing CT and MR images, CT and MR are represented by Gabors to get Gabor representation pairs firstly, each Gabor representation pair is put into the corresponding trained G- CNN for preliminary fusion, then use the fuzzy neural network to fuse multiple outputs of the G- CNNs to obtain the final fused image. Compared with the nine recent state-of-the-art multimodal fusion methods, the average mutual information of the three groups of experiments has increased by 13%, 10.3%, and 10% respectively; the average spatial frequency has increased by 10.3%, 20%, and 10.7%; the average standard deviation has increased respectively 12.4%, 10.8%, 14.4%; the average edge retention information increased by 33.5%, 22%, and 43%. The experimental results show that the proposed fusion method is significantly better than the other comparative fusion methods in objective evaluation and visual quality. It has the best performance on the four indicators and can better integrate the rich texture features and the clear edge information of the source images into the final fused image, which improves the quality of multimodal medical image fusion, and effectively assists doctors in disease diagnosis.
format	article
author	Lifang Wang Jin Zhang Yang Liu Jia Mi Jiong Zhang
author_facet	Lifang Wang Jin Zhang Yang Liu Jia Mi Jiong Zhang
author_sort	Lifang Wang
title	Multimodal Medical Image Fusion Based on Gabor Representation Combination of Multi-CNN and Fuzzy Neural Network
title_short	Multimodal Medical Image Fusion Based on Gabor Representation Combination of Multi-CNN and Fuzzy Neural Network
title_full	Multimodal Medical Image Fusion Based on Gabor Representation Combination of Multi-CNN and Fuzzy Neural Network
title_fullStr	Multimodal Medical Image Fusion Based on Gabor Representation Combination of Multi-CNN and Fuzzy Neural Network
title_full_unstemmed	Multimodal Medical Image Fusion Based on Gabor Representation Combination of Multi-CNN and Fuzzy Neural Network
title_sort	multimodal medical image fusion based on gabor representation combination of multi-cnn and fuzzy neural network
publisher	IEEE
publishDate	2021
url	https://doaj.org/article/c56b7a66d70d45ce82060e474d890372
work_keys_str_mv	AT lifangwang multimodalmedicalimagefusionbasedongaborrepresentationcombinationofmulticnnandfuzzyneuralnetwork AT jinzhang multimodalmedicalimagefusionbasedongaborrepresentationcombinationofmulticnnandfuzzyneuralnetwork AT yangliu multimodalmedicalimagefusionbasedongaborrepresentationcombinationofmulticnnandfuzzyneuralnetwork AT jiami multimodalmedicalimagefusionbasedongaborrepresentationcombinationofmulticnnandfuzzyneuralnetwork AT jiongzhang multimodalmedicalimagefusionbasedongaborrepresentationcombinationofmulticnnandfuzzyneuralnetwork
_version_	1718420634414874624

Multimodal Medical Image Fusion Based on Gabor Representation Combination of Multi-CNN and Fuzzy Neural Network

Ejemplares similares