An Energy-Efficient Edge Computing Paradigm for Convolution-Based Image Upsampling

State-of-the-art deep learning solutions for image upsampling are currently trained using either resize or sub-pixel convolution to learn kernels that generate high fidelity images with minimal artifacts. However, performing inference with these learned convolution kernels requires memory-intensive...

Descripción completa

Guardado en:

Detalles Bibliográficos
Autores principales:	Ian Colbert, Kenneth Kreutz-Delgado, Srinjoy Das
Formato:	article
Lenguaje:	EN
Publicado:	IEEE 2021
Materias:	Computer vision deconvolution deep learning edge computing energy efficiency image upsampling Electrical engineering. Electronics. Nuclear engineering TK1-9971
Acceso en línea:	https://doaj.org/article/6918181698ef4588b0398f020c59e7d9
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

id	oai:doaj.org-article:6918181698ef4588b0398f020c59e7d9
record_format	dspace
spelling	oai:doaj.org-article:6918181698ef4588b0398f020c59e7d92021-11-18T00:09:37ZAn Energy-Efficient Edge Computing Paradigm for Convolution-Based Image Upsampling2169-353610.1109/ACCESS.2021.3123938https://doaj.org/article/6918181698ef4588b0398f020c59e7d92021-01-01T00:00:00Zhttps://ieeexplore.ieee.org/document/9592768/https://doaj.org/toc/2169-3536State-of-the-art deep learning solutions for image upsampling are currently trained using either resize or sub-pixel convolution to learn kernels that generate high fidelity images with minimal artifacts. However, performing inference with these learned convolution kernels requires memory-intensive feature map transformations that dominate time and energy costs in real-time applications. To alleviate this pressure on memory bandwidth, we propose a novel energy-efficient edge computing paradigm that confines the use of resize or sub-pixel convolution to training in the cloud by transforming learned convolution kernels to deconvolution kernels before deploying them for inference as a functionally equivalent deconvolution. These kernel transformations, intended as a one-time cost when shifting from training to inference, enable a systems designer to use each algorithm in their optimal context by preserving the image fidelity learned when training in the cloud while minimizing data transfer penalties during inference at the edge. We compare the inference properties of these convolution-based image upsampling algorithms and introduce a novel deconvolution inference algorithm, which we refer to as REVD2. To demonstrate the benefits of our approach, we upsample images selected from the BSD300 dataset using a pre-trained single-image super resolution network provided by the PyTorch model zoo. Using quantitative models of incurred time and energy costs to analyze this deep neural network, we estimate that using REVD2 for inference at the edge improves system latency by <inline-formula> <tex-math notation="LaTeX">$2.1\times $ </tex-math></inline-formula> or <inline-formula> <tex-math notation="LaTeX">$2.8\times $ </tex-math></inline-formula> and energy efficiency by <inline-formula> <tex-math notation="LaTeX">$2.1\times $ </tex-math></inline-formula> or <inline-formula> <tex-math notation="LaTeX">$2.7\times $ </tex-math></inline-formula> when respectively compared to sub-pixel or resize convolution counterparts.Ian ColbertKenneth Kreutz-DelgadoSrinjoy DasIEEEarticleComputer visiondeconvolutiondeep learningedge computingenergy efficiencyimage upsamplingElectrical engineering. Electronics. Nuclear engineeringTK1-9971ENIEEE Access, Vol 9, Pp 147967-147984 (2021)
institution	DOAJ
collection	DOAJ
language	EN
topic	Computer vision deconvolution deep learning edge computing energy efficiency image upsampling Electrical engineering. Electronics. Nuclear engineering TK1-9971
spellingShingle	Computer vision deconvolution deep learning edge computing energy efficiency image upsampling Electrical engineering. Electronics. Nuclear engineering TK1-9971 Ian Colbert Kenneth Kreutz-Delgado Srinjoy Das An Energy-Efficient Edge Computing Paradigm for Convolution-Based Image Upsampling
description	State-of-the-art deep learning solutions for image upsampling are currently trained using either resize or sub-pixel convolution to learn kernels that generate high fidelity images with minimal artifacts. However, performing inference with these learned convolution kernels requires memory-intensive feature map transformations that dominate time and energy costs in real-time applications. To alleviate this pressure on memory bandwidth, we propose a novel energy-efficient edge computing paradigm that confines the use of resize or sub-pixel convolution to training in the cloud by transforming learned convolution kernels to deconvolution kernels before deploying them for inference as a functionally equivalent deconvolution. These kernel transformations, intended as a one-time cost when shifting from training to inference, enable a systems designer to use each algorithm in their optimal context by preserving the image fidelity learned when training in the cloud while minimizing data transfer penalties during inference at the edge. We compare the inference properties of these convolution-based image upsampling algorithms and introduce a novel deconvolution inference algorithm, which we refer to as REVD2. To demonstrate the benefits of our approach, we upsample images selected from the BSD300 dataset using a pre-trained single-image super resolution network provided by the PyTorch model zoo. Using quantitative models of incurred time and energy costs to analyze this deep neural network, we estimate that using REVD2 for inference at the edge improves system latency by <inline-formula> <tex-math notation="LaTeX">$2.1\times $ </tex-math></inline-formula> or <inline-formula> <tex-math notation="LaTeX">$2.8\times $ </tex-math></inline-formula> and energy efficiency by <inline-formula> <tex-math notation="LaTeX">$2.1\times $ </tex-math></inline-formula> or <inline-formula> <tex-math notation="LaTeX">$2.7\times $ </tex-math></inline-formula> when respectively compared to sub-pixel or resize convolution counterparts.
format	article
author	Ian Colbert Kenneth Kreutz-Delgado Srinjoy Das
author_facet	Ian Colbert Kenneth Kreutz-Delgado Srinjoy Das
author_sort	Ian Colbert
title	An Energy-Efficient Edge Computing Paradigm for Convolution-Based Image Upsampling
title_short	An Energy-Efficient Edge Computing Paradigm for Convolution-Based Image Upsampling
title_full	An Energy-Efficient Edge Computing Paradigm for Convolution-Based Image Upsampling
title_fullStr	An Energy-Efficient Edge Computing Paradigm for Convolution-Based Image Upsampling
title_full_unstemmed	An Energy-Efficient Edge Computing Paradigm for Convolution-Based Image Upsampling
title_sort	energy-efficient edge computing paradigm for convolution-based image upsampling
publisher	IEEE
publishDate	2021
url	https://doaj.org/article/6918181698ef4588b0398f020c59e7d9
work_keys_str_mv	AT iancolbert anenergyefficientedgecomputingparadigmforconvolutionbasedimageupsampling AT kennethkreutzdelgado anenergyefficientedgecomputingparadigmforconvolutionbasedimageupsampling AT srinjoydas anenergyefficientedgecomputingparadigmforconvolutionbasedimageupsampling AT iancolbert energyefficientedgecomputingparadigmforconvolutionbasedimageupsampling AT kennethkreutzdelgado energyefficientedgecomputingparadigmforconvolutionbasedimageupsampling AT srinjoydas energyefficientedgecomputingparadigmforconvolutionbasedimageupsampling
_version_	1718425228977111040

An Energy-Efficient Edge Computing Paradigm for Convolution-Based Image Upsampling

Ejemplares similares