Robust feature space separation for deep convolutional neural network training

Abstract This paper introduces two deep convolutional neural network training techniques that lead to more robust feature subspace separation in comparison to traditional training. Assume that dataset has M labels. The first method creates M deep convolutional neural networks called $$\{\text {DCNN}...

Descripción completa

Guardado en:

Detalles Bibliográficos
Autores principales:	Ali Sekmen, Mustafa Parlaktuna, Ayad Abdul-Malek, Erdem Erdemir, Ahmet Bugra Koku
Formato:	article
Lenguaje:	EN
Publicado:	Springer 2021
Materias:	Deep Convolutional Neural Networks Subspace Separation Robust Deep Learning Computational linguistics. Natural language processing P98-98.5 Electronic computers. Computer science QA75.5-76.95
Acceso en línea:	https://doaj.org/article/0c4975ea7511433cbff1e6d106499b8e
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

id	oai:doaj.org-article:0c4975ea7511433cbff1e6d106499b8e
record_format	dspace
spelling	oai:doaj.org-article:0c4975ea7511433cbff1e6d106499b8e2021-11-21T12:31:43ZRobust feature space separation for deep convolutional neural network training10.1007/s44163-021-00013-12731-0809https://doaj.org/article/0c4975ea7511433cbff1e6d106499b8e2021-11-01T00:00:00Zhttps://doi.org/10.1007/s44163-021-00013-1https://doaj.org/toc/2731-0809Abstract This paper introduces two deep convolutional neural network training techniques that lead to more robust feature subspace separation in comparison to traditional training. Assume that dataset has M labels. The first method creates M deep convolutional neural networks called $$\{\text {DCNN}_i\}_{i=1}^{M}$$ { DCNN i } i = 1 M . Each of the networks $$\text {DCNN}_i$$ DCNN i is composed of a convolutional neural network ( $$\text {CNN}_i$$ CNN i ) and a fully connected neural network ( $$\text {FCNN}_i$$ FCNN i ). In training, a set of projection matrices $$\{\mathbf {P}_i\}_{i=1}^M$$ { P i } i = 1 M are created and adaptively updated as representations for feature subspaces $$\{\mathcal {S}_i\}_{i=1}^M$$ { S i } i = 1 M . A rejection value is computed for each training based on its projections on feature subspaces. Each $$\text {FCNN}_i$$ FCNN i acts as a binary classifier with a cost function whose main parameter is rejection values. A threshold value $$t_i$$ t i is determined for $$i^{th}$$ i th network $$\text {DCNN}_i$$ DCNN i . A testing strategy utilizing $$\{t_i\}_{i=1}^M$$ { t i } i = 1 M is also introduced. The second method creates a single DCNN and it computes a cost function whose parameters depend on subspace separations using the geodesic distance on the Grasmannian manifold of subspaces $$\mathcal {S}_i$$ S i and the sum of all remaining subspaces $$\{\mathcal {S}_j\}_{j=1,j\ne i}^M$$ { S j } j = 1 , j ≠ i M . The proposed methods are tested using multiple network topologies. It is shown that while the first method works better for smaller networks, the second method performs better for complex architectures.Ali SekmenMustafa ParlaktunaAyad Abdul-MalekErdem ErdemirAhmet Bugra KokuSpringerarticleDeep Convolutional Neural NetworksSubspace SeparationRobust Deep LearningComputational linguistics. Natural language processingP98-98.5Electronic computers. Computer scienceQA75.5-76.95ENDiscover Artificial Intelligence, Vol 1, Iss 1, Pp 1-11 (2021)
institution	DOAJ
collection	DOAJ
language	EN
topic	Deep Convolutional Neural Networks Subspace Separation Robust Deep Learning Computational linguistics. Natural language processing P98-98.5 Electronic computers. Computer science QA75.5-76.95
spellingShingle	Deep Convolutional Neural Networks Subspace Separation Robust Deep Learning Computational linguistics. Natural language processing P98-98.5 Electronic computers. Computer science QA75.5-76.95 Ali Sekmen Mustafa Parlaktuna Ayad Abdul-Malek Erdem Erdemir Ahmet Bugra Koku Robust feature space separation for deep convolutional neural network training
description	Abstract This paper introduces two deep convolutional neural network training techniques that lead to more robust feature subspace separation in comparison to traditional training. Assume that dataset has M labels. The first method creates M deep convolutional neural networks called $$\{\text {DCNN}_i\}_{i=1}^{M}$$ { DCNN i } i = 1 M . Each of the networks $$\text {DCNN}_i$$ DCNN i is composed of a convolutional neural network ( $$\text {CNN}_i$$ CNN i ) and a fully connected neural network ( $$\text {FCNN}_i$$ FCNN i ). In training, a set of projection matrices $$\{\mathbf {P}_i\}_{i=1}^M$$ { P i } i = 1 M are created and adaptively updated as representations for feature subspaces $$\{\mathcal {S}_i\}_{i=1}^M$$ { S i } i = 1 M . A rejection value is computed for each training based on its projections on feature subspaces. Each $$\text {FCNN}_i$$ FCNN i acts as a binary classifier with a cost function whose main parameter is rejection values. A threshold value $$t_i$$ t i is determined for $$i^{th}$$ i th network $$\text {DCNN}_i$$ DCNN i . A testing strategy utilizing $$\{t_i\}_{i=1}^M$$ { t i } i = 1 M is also introduced. The second method creates a single DCNN and it computes a cost function whose parameters depend on subspace separations using the geodesic distance on the Grasmannian manifold of subspaces $$\mathcal {S}_i$$ S i and the sum of all remaining subspaces $$\{\mathcal {S}_j\}_{j=1,j\ne i}^M$$ { S j } j = 1 , j ≠ i M . The proposed methods are tested using multiple network topologies. It is shown that while the first method works better for smaller networks, the second method performs better for complex architectures.
format	article
author	Ali Sekmen Mustafa Parlaktuna Ayad Abdul-Malek Erdem Erdemir Ahmet Bugra Koku
author_facet	Ali Sekmen Mustafa Parlaktuna Ayad Abdul-Malek Erdem Erdemir Ahmet Bugra Koku
author_sort	Ali Sekmen
title	Robust feature space separation for deep convolutional neural network training
title_short	Robust feature space separation for deep convolutional neural network training
title_full	Robust feature space separation for deep convolutional neural network training
title_fullStr	Robust feature space separation for deep convolutional neural network training
title_full_unstemmed	Robust feature space separation for deep convolutional neural network training
title_sort	robust feature space separation for deep convolutional neural network training
publisher	Springer
publishDate	2021
url	https://doaj.org/article/0c4975ea7511433cbff1e6d106499b8e
work_keys_str_mv	AT alisekmen robustfeaturespaceseparationfordeepconvolutionalneuralnetworktraining AT mustafaparlaktuna robustfeaturespaceseparationfordeepconvolutionalneuralnetworktraining AT ayadabdulmalek robustfeaturespaceseparationfordeepconvolutionalneuralnetworktraining AT erdemerdemir robustfeaturespaceseparationfordeepconvolutionalneuralnetworktraining AT ahmetbugrakoku robustfeaturespaceseparationfordeepconvolutionalneuralnetworktraining
_version_	1718418926690369536

Robust feature space separation for deep convolutional neural network training

Ejemplares similares