360° Image Reference-Based Super-Resolution Using Latitude-Aware Convolution Learned From Synthetic to Real
High-resolution (HR) 360° images offer great advantages wherever an omnidirectional view is necessary such as in autonomous robot systems and virtual reality (VR) applications. One or more 360° images in adjacent views can be utilized to significantly improve the resolution of...
Guardado en:
| Autores principales: | , , |
|---|---|
| Formato: | article |
| Lenguaje: | EN |
| Publicado: |
IEEE
2021
|
| Materias: | |
| Acceso en línea: | https://doaj.org/article/ede04c61b82e43e3af446cc5ee9c23d9 |
| Etiquetas: |
Agregar Etiqueta
Sin Etiquetas, Sea el primero en etiquetar este registro!
|
| id |
oai:doaj.org-article:ede04c61b82e43e3af446cc5ee9c23d9 |
|---|---|
| record_format |
dspace |
| spelling |
oai:doaj.org-article:ede04c61b82e43e3af446cc5ee9c23d92021-12-01T00:00:54Z360° Image Reference-Based Super-Resolution Using Latitude-Aware Convolution Learned From Synthetic to Real2169-353610.1109/ACCESS.2021.3128574https://doaj.org/article/ede04c61b82e43e3af446cc5ee9c23d92021-01-01T00:00:00Zhttps://ieeexplore.ieee.org/document/9617634/https://doaj.org/toc/2169-3536High-resolution (HR) 360° images offer great advantages wherever an omnidirectional view is necessary such as in autonomous robot systems and virtual reality (VR) applications. One or more 360° images in adjacent views can be utilized to significantly improve the resolution of a target 360° image. In this paper, we propose an efficient reference-based 360° image super-resolution (RefSR) technique to exploit a wide field of view (FoV) among adjacent 360° cameras. Effective exploitation of spatial correlation is critical to achieving high quality even though the distortion inherent in the equi-rectangular projection (ERP) is a nontrivial problem. Accordingly, we develop a long-range 360 disparity estimator (DE360) to overcome a large and distorted disparity, particularly near the poles. Latitude-aware convolution (LatConv) is designed to generate more robust features to circumvent the distortion and keep the image quality. We also develop synthetic 360° image datasets and introduce a synthetic-to-real learning scheme that transfers knowledge learned from synthetic 360° images to a deep neural network conducting super-resolution (SR) of camera-captured images. The proposed network can learn useful features in the ERP-domain using a sufficient number of synthetic samples. The network is then adapted to camera-captured images through the transfer layer with a limited number of real-world datasets.Hee-Jae KimJe-Won KangByung-Uk LeeIEEEarticle360° imageryreference-based super-resolutionlatitude-aware convolutiondisparity estimationsynthetic-to-real transfer learningElectrical engineering. Electronics. Nuclear engineeringTK1-9971ENIEEE Access, Vol 9, Pp 155924-155935 (2021) |
| institution |
DOAJ |
| collection |
DOAJ |
| language |
EN |
| topic |
360° imagery reference-based super-resolution latitude-aware convolution disparity estimation synthetic-to-real transfer learning Electrical engineering. Electronics. Nuclear engineering TK1-9971 |
| spellingShingle |
360° imagery reference-based super-resolution latitude-aware convolution disparity estimation synthetic-to-real transfer learning Electrical engineering. Electronics. Nuclear engineering TK1-9971 Hee-Jae Kim Je-Won Kang Byung-Uk Lee 360° Image Reference-Based Super-Resolution Using Latitude-Aware Convolution Learned From Synthetic to Real |
| description |
High-resolution (HR) 360° images offer great advantages wherever an omnidirectional view is necessary such as in autonomous robot systems and virtual reality (VR) applications. One or more 360° images in adjacent views can be utilized to significantly improve the resolution of a target 360° image. In this paper, we propose an efficient reference-based 360° image super-resolution (RefSR) technique to exploit a wide field of view (FoV) among adjacent 360° cameras. Effective exploitation of spatial correlation is critical to achieving high quality even though the distortion inherent in the equi-rectangular projection (ERP) is a nontrivial problem. Accordingly, we develop a long-range 360 disparity estimator (DE360) to overcome a large and distorted disparity, particularly near the poles. Latitude-aware convolution (LatConv) is designed to generate more robust features to circumvent the distortion and keep the image quality. We also develop synthetic 360° image datasets and introduce a synthetic-to-real learning scheme that transfers knowledge learned from synthetic 360° images to a deep neural network conducting super-resolution (SR) of camera-captured images. The proposed network can learn useful features in the ERP-domain using a sufficient number of synthetic samples. The network is then adapted to camera-captured images through the transfer layer with a limited number of real-world datasets. |
| format |
article |
| author |
Hee-Jae Kim Je-Won Kang Byung-Uk Lee |
| author_facet |
Hee-Jae Kim Je-Won Kang Byung-Uk Lee |
| author_sort |
Hee-Jae Kim |
| title |
360° Image Reference-Based Super-Resolution Using Latitude-Aware Convolution Learned From Synthetic to Real |
| title_short |
360° Image Reference-Based Super-Resolution Using Latitude-Aware Convolution Learned From Synthetic to Real |
| title_full |
360° Image Reference-Based Super-Resolution Using Latitude-Aware Convolution Learned From Synthetic to Real |
| title_fullStr |
360° Image Reference-Based Super-Resolution Using Latitude-Aware Convolution Learned From Synthetic to Real |
| title_full_unstemmed |
360° Image Reference-Based Super-Resolution Using Latitude-Aware Convolution Learned From Synthetic to Real |
| title_sort |
360° image reference-based super-resolution using latitude-aware convolution learned from synthetic to real |
| publisher |
IEEE |
| publishDate |
2021 |
| url |
https://doaj.org/article/ede04c61b82e43e3af446cc5ee9c23d9 |
| work_keys_str_mv |
AT heejaekim 360x00b0imagereferencebasedsuperresolutionusinglatitudeawareconvolutionlearnedfromsynthetictoreal AT jewonkang 360x00b0imagereferencebasedsuperresolutionusinglatitudeawareconvolutionlearnedfromsynthetictoreal AT byunguklee 360x00b0imagereferencebasedsuperresolutionusinglatitudeawareconvolutionlearnedfromsynthetictoreal |
| _version_ |
1718406174668226560 |