Crossing the Reality Gap: A Survey on Sim-to-Real Transferability of Robot Controllers in Reinforcement Learning

The growing demand for robots able to act autonomously in complex scenarios has widely accelerated the introduction of Reinforcement Learning (RL) in robots control applications. However, the <italic>trial and error</italic> intrinsic nature of RL may result in long training time on real...

Descripción completa

Guardado en:

Detalles Bibliográficos
Autores principales:	Erica Salvato, Gianfranco Fenu, Eric Medvet, Felice Andrea Pellegrino
Formato:	article
Lenguaje:	EN
Publicado:	IEEE 2021
Materias:	Reality gap reinforcement learning robotics sim-to-real Electrical engineering. Electronics. Nuclear engineering TK1-9971
Acceso en línea:	https://doaj.org/article/9f38aed8c13146a8bbfadadf1817d86b
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

id	oai:doaj.org-article:9f38aed8c13146a8bbfadadf1817d86b
record_format	dspace
spelling	oai:doaj.org-article:9f38aed8c13146a8bbfadadf1817d86b2021-11-20T00:01:56ZCrossing the Reality Gap: A Survey on Sim-to-Real Transferability of Robot Controllers in Reinforcement Learning2169-353610.1109/ACCESS.2021.3126658https://doaj.org/article/9f38aed8c13146a8bbfadadf1817d86b2021-01-01T00:00:00Zhttps://ieeexplore.ieee.org/document/9606868/https://doaj.org/toc/2169-3536The growing demand for robots able to act autonomously in complex scenarios has widely accelerated the introduction of Reinforcement Learning (RL) in robots control applications. However, the <italic>trial and error</italic> intrinsic nature of RL may result in long training time on real robots and, moreover, it may lead to dangerous outcomes. While simulators are useful tools to accelerate RL training and to ensure safety, they often are provided only with an approximated model of robot dynamics and of its interaction with the surrounding environment, thus resulting in what is called the <italic>reality gap</italic> (RG): a mismatch of simulated and real control-law performances caused by the inaccurate representation of the real environment in simulation. The most undesirable result occurs when the controller learnt in simulation fails the task on the real robot, thus resulting in an unsuccessful <italic>sim-to-real</italic> transfer. The goal of the present survey is threefold: (1) to identify the main approaches to face the RG problem in the context of robot control with RL, (2) to point out their shortcomings, and (3) to outline new potential research areas.Erica SalvatoGianfranco FenuEric MedvetFelice Andrea PellegrinoIEEEarticleReality gapreinforcement learningroboticssim-to-realElectrical engineering. Electronics. Nuclear engineeringTK1-9971ENIEEE Access, Vol 9, Pp 153171-153187 (2021)
institution	DOAJ
collection	DOAJ
language	EN
topic	Reality gap reinforcement learning robotics sim-to-real Electrical engineering. Electronics. Nuclear engineering TK1-9971
spellingShingle	Reality gap reinforcement learning robotics sim-to-real Electrical engineering. Electronics. Nuclear engineering TK1-9971 Erica Salvato Gianfranco Fenu Eric Medvet Felice Andrea Pellegrino Crossing the Reality Gap: A Survey on Sim-to-Real Transferability of Robot Controllers in Reinforcement Learning
description	The growing demand for robots able to act autonomously in complex scenarios has widely accelerated the introduction of Reinforcement Learning (RL) in robots control applications. However, the <italic>trial and error</italic> intrinsic nature of RL may result in long training time on real robots and, moreover, it may lead to dangerous outcomes. While simulators are useful tools to accelerate RL training and to ensure safety, they often are provided only with an approximated model of robot dynamics and of its interaction with the surrounding environment, thus resulting in what is called the <italic>reality gap</italic> (RG): a mismatch of simulated and real control-law performances caused by the inaccurate representation of the real environment in simulation. The most undesirable result occurs when the controller learnt in simulation fails the task on the real robot, thus resulting in an unsuccessful <italic>sim-to-real</italic> transfer. The goal of the present survey is threefold: (1) to identify the main approaches to face the RG problem in the context of robot control with RL, (2) to point out their shortcomings, and (3) to outline new potential research areas.
format	article
author	Erica Salvato Gianfranco Fenu Eric Medvet Felice Andrea Pellegrino
author_facet	Erica Salvato Gianfranco Fenu Eric Medvet Felice Andrea Pellegrino
author_sort	Erica Salvato
title	Crossing the Reality Gap: A Survey on Sim-to-Real Transferability of Robot Controllers in Reinforcement Learning
title_short	Crossing the Reality Gap: A Survey on Sim-to-Real Transferability of Robot Controllers in Reinforcement Learning
title_full	Crossing the Reality Gap: A Survey on Sim-to-Real Transferability of Robot Controllers in Reinforcement Learning
title_fullStr	Crossing the Reality Gap: A Survey on Sim-to-Real Transferability of Robot Controllers in Reinforcement Learning
title_full_unstemmed	Crossing the Reality Gap: A Survey on Sim-to-Real Transferability of Robot Controllers in Reinforcement Learning
title_sort	crossing the reality gap: a survey on sim-to-real transferability of robot controllers in reinforcement learning
publisher	IEEE
publishDate	2021
url	https://doaj.org/article/9f38aed8c13146a8bbfadadf1817d86b
work_keys_str_mv	AT ericasalvato crossingtherealitygapasurveyonsimtorealtransferabilityofrobotcontrollersinreinforcementlearning AT gianfrancofenu crossingtherealitygapasurveyonsimtorealtransferabilityofrobotcontrollersinreinforcementlearning AT ericmedvet crossingtherealitygapasurveyonsimtorealtransferabilityofrobotcontrollersinreinforcementlearning AT feliceandreapellegrino crossingtherealitygapasurveyonsimtorealtransferabilityofrobotcontrollersinreinforcementlearning
_version_	1718419852821004288

Crossing the Reality Gap: A Survey on Sim-to-Real Transferability of Robot Controllers in Reinforcement Learning

Ejemplares similares