Reinforcement learning control of a biomechanical model of the upper extremity

Abstract Among the infinite number of possible movements that can be produced, humans are commonly assumed to choose those that optimize criteria such as minimizing movement time, subject to certain movement constraints like signal-dependent and constant motor noise. While so far these assumptions h...

Descripción completa

Guardado en:

Detalles Bibliográficos
Autores principales:	Florian Fischer, Miroslav Bachinski, Markus Klar, Arthur Fleig, Jörg Müller
Formato:	article
Lenguaje:	EN
Publicado:	Nature Portfolio 2021
Materias:	Medicine R Science Q
Acceso en línea:	https://doaj.org/article/33d3fa44c29842128b53c1355182dbf9
Etiquetas:	Agregar Etiqueta Sin Etiquetas, Sea el primero en etiquetar este registro!

id	oai:doaj.org-article:33d3fa44c29842128b53c1355182dbf9
record_format	dspace
spelling	oai:doaj.org-article:33d3fa44c29842128b53c1355182dbf92021-12-02T16:14:10ZReinforcement learning control of a biomechanical model of the upper extremity10.1038/s41598-021-93760-12045-2322https://doaj.org/article/33d3fa44c29842128b53c1355182dbf92021-07-01T00:00:00Zhttps://doi.org/10.1038/s41598-021-93760-1https://doaj.org/toc/2045-2322Abstract Among the infinite number of possible movements that can be produced, humans are commonly assumed to choose those that optimize criteria such as minimizing movement time, subject to certain movement constraints like signal-dependent and constant motor noise. While so far these assumptions have only been evaluated for simplified point-mass or planar models, we address the question of whether they can predict reaching movements in a full skeletal model of the human upper extremity. We learn a control policy using a motor babbling approach as implemented in reinforcement learning, using aimed movements of the tip of the right index finger towards randomly placed 3D targets of varying size. We use a state-of-the-art biomechanical model, which includes seven actuated degrees of freedom. To deal with the curse of dimensionality, we use a simplified second-order muscle model, acting at each degree of freedom instead of individual muscles. The results confirm that the assumptions of signal-dependent and constant motor noise, together with the objective of movement time minimization, are sufficient for a state-of-the-art skeletal model of the human upper extremity to reproduce complex phenomena of human movement, in particular Fitts’ Law and the $$\frac{2}{3}$$ 2 3 Power Law. This result supports the notion that control of the complex human biomechanical system can plausibly be determined by a set of simple assumptions and can easily be learned.Florian FischerMiroslav BachinskiMarkus KlarArthur FleigJörg MüllerNature PortfolioarticleMedicineRScienceQENScientific Reports, Vol 11, Iss 1, Pp 1-15 (2021)
institution	DOAJ
collection	DOAJ
language	EN
topic	Medicine R Science Q
spellingShingle	Medicine R Science Q Florian Fischer Miroslav Bachinski Markus Klar Arthur Fleig Jörg Müller Reinforcement learning control of a biomechanical model of the upper extremity
description	Abstract Among the infinite number of possible movements that can be produced, humans are commonly assumed to choose those that optimize criteria such as minimizing movement time, subject to certain movement constraints like signal-dependent and constant motor noise. While so far these assumptions have only been evaluated for simplified point-mass or planar models, we address the question of whether they can predict reaching movements in a full skeletal model of the human upper extremity. We learn a control policy using a motor babbling approach as implemented in reinforcement learning, using aimed movements of the tip of the right index finger towards randomly placed 3D targets of varying size. We use a state-of-the-art biomechanical model, which includes seven actuated degrees of freedom. To deal with the curse of dimensionality, we use a simplified second-order muscle model, acting at each degree of freedom instead of individual muscles. The results confirm that the assumptions of signal-dependent and constant motor noise, together with the objective of movement time minimization, are sufficient for a state-of-the-art skeletal model of the human upper extremity to reproduce complex phenomena of human movement, in particular Fitts’ Law and the $$\frac{2}{3}$$ 2 3 Power Law. This result supports the notion that control of the complex human biomechanical system can plausibly be determined by a set of simple assumptions and can easily be learned.
format	article
author	Florian Fischer Miroslav Bachinski Markus Klar Arthur Fleig Jörg Müller
author_facet	Florian Fischer Miroslav Bachinski Markus Klar Arthur Fleig Jörg Müller
author_sort	Florian Fischer
title	Reinforcement learning control of a biomechanical model of the upper extremity
title_short	Reinforcement learning control of a biomechanical model of the upper extremity
title_full	Reinforcement learning control of a biomechanical model of the upper extremity
title_fullStr	Reinforcement learning control of a biomechanical model of the upper extremity
title_full_unstemmed	Reinforcement learning control of a biomechanical model of the upper extremity
title_sort	reinforcement learning control of a biomechanical model of the upper extremity
publisher	Nature Portfolio
publishDate	2021
url	https://doaj.org/article/33d3fa44c29842128b53c1355182dbf9
work_keys_str_mv	AT florianfischer reinforcementlearningcontrolofabiomechanicalmodeloftheupperextremity AT miroslavbachinski reinforcementlearningcontrolofabiomechanicalmodeloftheupperextremity AT markusklar reinforcementlearningcontrolofabiomechanicalmodeloftheupperextremity AT arthurfleig reinforcementlearningcontrolofabiomechanicalmodeloftheupperextremity AT jorgmuller reinforcementlearningcontrolofabiomechanicalmodeloftheupperextremity
_version_	1718384342762258432

Reinforcement learning control of a biomechanical model of the upper extremity

Ejemplares similares