Knowledge Distillation of Grassmann Manifold Network for Remote Sensing Scene Classification

Due to device limitations, small networks are necessary for some real-world scenarios, such as satellites and micro-robots. Therefore, the development of a network with both good performance and small size is an important area of research. Deep networks can learn well from large amounts of data, whi...

Descripción completa

Guardado en:

Detalles Bibliográficos
Autores principales:	Ling Tian, Zhichao Wang, Bokun He, Chu He, Dingwen Wang, Deshi Li
Formato:	article
Lenguaje:	EN
Publicado:	MDPI AG 2021
Materias:	knowledge distillation Grassmann manifold neural network Science Q
Acceso en línea:	https://doaj.org/article/b502d9b230bd4486a4c0a2f5b86bebe2
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

id	oai:doaj.org-article:b502d9b230bd4486a4c0a2f5b86bebe2
record_format	dspace
spelling	oai:doaj.org-article:b502d9b230bd4486a4c0a2f5b86bebe22021-11-25T18:54:01ZKnowledge Distillation of Grassmann Manifold Network for Remote Sensing Scene Classification10.3390/rs132245372072-4292https://doaj.org/article/b502d9b230bd4486a4c0a2f5b86bebe22021-11-01T00:00:00Zhttps://www.mdpi.com/2072-4292/13/22/4537https://doaj.org/toc/2072-4292Due to device limitations, small networks are necessary for some real-world scenarios, such as satellites and micro-robots. Therefore, the development of a network with both good performance and small size is an important area of research. Deep networks can learn well from large amounts of data, while manifold networks have outstanding feature representation at small sizes. In this paper, we propose an approach that exploits the advantages of deep networks and shallow Grassmannian manifold networks. Inspired by knowledge distillation, we use the information learned from convolutional neural networks to guide the training of the manifold networks. Our approach leads to a reduction in model size, which addresses the problem of deploying deep learning on resource-limited embedded devices. Finally, a series of experiments were conducted on four remote sensing scene classification datasets. The method in this paper improved the classification accuracy by 2.31% and 1.73% on the UC Merced Land Use and SIRIWHU datasets, respectively, and the experimental results demonstrate the effectiveness of our approach.Ling TianZhichao WangBokun HeChu HeDingwen WangDeshi LiMDPI AGarticleknowledge distillationGrassmann manifoldneural networkScienceQENRemote Sensing, Vol 13, Iss 4537, p 4537 (2021)
institution	DOAJ
collection	DOAJ
language	EN
topic	knowledge distillation Grassmann manifold neural network Science Q
spellingShingle	knowledge distillation Grassmann manifold neural network Science Q Ling Tian Zhichao Wang Bokun He Chu He Dingwen Wang Deshi Li Knowledge Distillation of Grassmann Manifold Network for Remote Sensing Scene Classification
description	Due to device limitations, small networks are necessary for some real-world scenarios, such as satellites and micro-robots. Therefore, the development of a network with both good performance and small size is an important area of research. Deep networks can learn well from large amounts of data, while manifold networks have outstanding feature representation at small sizes. In this paper, we propose an approach that exploits the advantages of deep networks and shallow Grassmannian manifold networks. Inspired by knowledge distillation, we use the information learned from convolutional neural networks to guide the training of the manifold networks. Our approach leads to a reduction in model size, which addresses the problem of deploying deep learning on resource-limited embedded devices. Finally, a series of experiments were conducted on four remote sensing scene classification datasets. The method in this paper improved the classification accuracy by 2.31% and 1.73% on the UC Merced Land Use and SIRIWHU datasets, respectively, and the experimental results demonstrate the effectiveness of our approach.
format	article
author	Ling Tian Zhichao Wang Bokun He Chu He Dingwen Wang Deshi Li
author_facet	Ling Tian Zhichao Wang Bokun He Chu He Dingwen Wang Deshi Li
author_sort	Ling Tian
title	Knowledge Distillation of Grassmann Manifold Network for Remote Sensing Scene Classification
title_short	Knowledge Distillation of Grassmann Manifold Network for Remote Sensing Scene Classification
title_full	Knowledge Distillation of Grassmann Manifold Network for Remote Sensing Scene Classification
title_fullStr	Knowledge Distillation of Grassmann Manifold Network for Remote Sensing Scene Classification
title_full_unstemmed	Knowledge Distillation of Grassmann Manifold Network for Remote Sensing Scene Classification
title_sort	knowledge distillation of grassmann manifold network for remote sensing scene classification
publisher	MDPI AG
publishDate	2021
url	https://doaj.org/article/b502d9b230bd4486a4c0a2f5b86bebe2
work_keys_str_mv	AT lingtian knowledgedistillationofgrassmannmanifoldnetworkforremotesensingsceneclassification AT zhichaowang knowledgedistillationofgrassmannmanifoldnetworkforremotesensingsceneclassification AT bokunhe knowledgedistillationofgrassmannmanifoldnetworkforremotesensingsceneclassification AT chuhe knowledgedistillationofgrassmannmanifoldnetworkforremotesensingsceneclassification AT dingwenwang knowledgedistillationofgrassmannmanifoldnetworkforremotesensingsceneclassification AT deshili knowledgedistillationofgrassmannmanifoldnetworkforremotesensingsceneclassification
_version_	1718410613941600256

Knowledge Distillation of Grassmann Manifold Network for Remote Sensing Scene Classification

Ejemplares similares