Real-Time Depth Video-Based Rendering for 6-DoF HMD Navigation and Light Field Displays

This paper presents a novel approach to provide immersive free navigation with 6 Degrees of Freedom in real-time for natural and virtual scenery, for both static and dynamic content. Stemming from the state-of-the-art in Depth Image-Based Rendering and the OpenGL pipeline, this new View Synthesis me...

Descripción completa

Guardado en:

Detalles Bibliográficos
Autores principales:	Daniele Bonatto, Sarah Fachada, Segolene Rogge, Adrian Munteanu, Gauthier Lafruit
Formato:	article
Lenguaje:	EN
Publicado:	IEEE 2021
Materias:	Virtual reality stereo image processing stereo vision free viewpoint navigation reference view synthesizer real-time view synthesis Electrical engineering. Electronics. Nuclear engineering TK1-9971
Acceso en línea:	https://doaj.org/article/f52a691319474da299d758aff8137569
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

id	oai:doaj.org-article:f52a691319474da299d758aff8137569
record_format	dspace
spelling	oai:doaj.org-article:f52a691319474da299d758aff81375692021-11-09T00:03:36ZReal-Time Depth Video-Based Rendering for 6-DoF HMD Navigation and Light Field Displays2169-353610.1109/ACCESS.2021.3123529https://doaj.org/article/f52a691319474da299d758aff81375692021-01-01T00:00:00Zhttps://ieeexplore.ieee.org/document/9590541/https://doaj.org/toc/2169-3536This paper presents a novel approach to provide immersive free navigation with 6 Degrees of Freedom in real-time for natural and virtual scenery, for both static and dynamic content. Stemming from the state-of-the-art in Depth Image-Based Rendering and the OpenGL pipeline, this new View Synthesis method achieves free navigation at up to 90 FPS and can take any number of input views with their corresponding depth maps as priors. Video content can be played thanks to GPU decompression, supporting free navigation with full parallax in real-time. To render a novel viewpoint, each selected input view is warped using the camera pose and associated depth map, using an implicit 3D representation. The warped views are then blended all together to generate the chosen virtual view. Various view blending approaches specifically designed to avoid visual artifacts are compared. Using as few as four input views appears to be an optimal trade-off between computation time and quality, allowing to synthesize high-quality stereoscopic views in real-time, offering a genuine immersive virtual reality experience. Additionally, the proposed approach provides high-quality rendering of a 3D scenery on holographic light field displays. Our results are comparable - objectively and subjectively - to the state of the art view synthesis tools NeRF and LLFF, while maintaining an overall lower complexity and real-time rendering.Daniele BonattoSarah FachadaSegolene RoggeAdrian MunteanuGauthier LafruitIEEEarticleVirtual realitystereo image processingstereo visionfree viewpoint navigationreference view synthesizerreal-time view synthesisElectrical engineering. Electronics. Nuclear engineeringTK1-9971ENIEEE Access, Vol 9, Pp 146868-146887 (2021)
institution	DOAJ
collection	DOAJ
language	EN
topic	Virtual reality stereo image processing stereo vision free viewpoint navigation reference view synthesizer real-time view synthesis Electrical engineering. Electronics. Nuclear engineering TK1-9971
spellingShingle	Virtual reality stereo image processing stereo vision free viewpoint navigation reference view synthesizer real-time view synthesis Electrical engineering. Electronics. Nuclear engineering TK1-9971 Daniele Bonatto Sarah Fachada Segolene Rogge Adrian Munteanu Gauthier Lafruit Real-Time Depth Video-Based Rendering for 6-DoF HMD Navigation and Light Field Displays
description	This paper presents a novel approach to provide immersive free navigation with 6 Degrees of Freedom in real-time for natural and virtual scenery, for both static and dynamic content. Stemming from the state-of-the-art in Depth Image-Based Rendering and the OpenGL pipeline, this new View Synthesis method achieves free navigation at up to 90 FPS and can take any number of input views with their corresponding depth maps as priors. Video content can be played thanks to GPU decompression, supporting free navigation with full parallax in real-time. To render a novel viewpoint, each selected input view is warped using the camera pose and associated depth map, using an implicit 3D representation. The warped views are then blended all together to generate the chosen virtual view. Various view blending approaches specifically designed to avoid visual artifacts are compared. Using as few as four input views appears to be an optimal trade-off between computation time and quality, allowing to synthesize high-quality stereoscopic views in real-time, offering a genuine immersive virtual reality experience. Additionally, the proposed approach provides high-quality rendering of a 3D scenery on holographic light field displays. Our results are comparable - objectively and subjectively - to the state of the art view synthesis tools NeRF and LLFF, while maintaining an overall lower complexity and real-time rendering.
format	article
author	Daniele Bonatto Sarah Fachada Segolene Rogge Adrian Munteanu Gauthier Lafruit
author_facet	Daniele Bonatto Sarah Fachada Segolene Rogge Adrian Munteanu Gauthier Lafruit
author_sort	Daniele Bonatto
title	Real-Time Depth Video-Based Rendering for 6-DoF HMD Navigation and Light Field Displays
title_short	Real-Time Depth Video-Based Rendering for 6-DoF HMD Navigation and Light Field Displays
title_full	Real-Time Depth Video-Based Rendering for 6-DoF HMD Navigation and Light Field Displays
title_fullStr	Real-Time Depth Video-Based Rendering for 6-DoF HMD Navigation and Light Field Displays
title_full_unstemmed	Real-Time Depth Video-Based Rendering for 6-DoF HMD Navigation and Light Field Displays
title_sort	real-time depth video-based rendering for 6-dof hmd navigation and light field displays
publisher	IEEE
publishDate	2021
url	https://doaj.org/article/f52a691319474da299d758aff8137569
work_keys_str_mv	AT danielebonatto realtimedepthvideobasedrenderingfor6dofhmdnavigationandlightfielddisplays AT sarahfachada realtimedepthvideobasedrenderingfor6dofhmdnavigationandlightfielddisplays AT segolenerogge realtimedepthvideobasedrenderingfor6dofhmdnavigationandlightfielddisplays AT adrianmunteanu realtimedepthvideobasedrenderingfor6dofhmdnavigationandlightfielddisplays AT gauthierlafruit realtimedepthvideobasedrenderingfor6dofhmdnavigationandlightfielddisplays
_version_	1718441440955072512

Real-Time Depth Video-Based Rendering for 6-DoF HMD Navigation and Light Field Displays

Ejemplares similares